Just another fine pipeline language
This project is in an early, alpha state. YMMV. Here there be dragons. Etc.
Getting started
If you’re new to JAFPL, begin by reading the overview on this page. then read about pipelines. To become familiar with the APIs provided by this project, read the APIs pages.
If you find errors in this documentation, or if you think there are parts that are unclear or simply missing, please file an issue to let us know.
What is it?
This project is a JVM library for evaluating arbitrary data flow graphs or “pipelines”. The nodes, or “steps”, in the graph are operations. The edges are how data flows between steps. A step takes its inputs, performs some arbitrary computation, and produces its outputs.
A simple, two step pipeline might be visualized with boxes and arrows.
This library doesn’t care what each step does and it doesn’t care what kind of data flows between steps. You construct a graph with edges. You pour data in at the top, stuff happens, data pours out the bottom.
Many steps are atomic in nature, they are “black boxes” of computation, but some naturally operate on subgraphs, these are “containers”. The following containers are supported:
- Groups
A group is the simplest possible container. It’s vacuous, providing no additional semantics.
- Iteration
Several flavors of iteration container are supported: iteration over a sequence, iteration while a condition is true, iteration to a fixed point, and iteration a specific number of times.
- Viewports
A viewport operates on subsections of its input, for example, on the values of specific keys in a JSON object or on specified descendants in an XML document. But equally, it could operate on regions of an image or lines of a text file.
- Conditionals
A conditional selects one of a set of possible subpipelines, based on a condition evaluated at runtime.
- Try/catch/finally error handling
Step implementations may encounter errors and raise exceptions. A try/catch container gives the pipeline author an opportunity to catch those errors and react accordingly.
It would be nice to generalize the abstraction of “container” to the point where a small number of primitives in the core library could be used to build arbitrary containers. So far, that goal seems elusive.
Where is it?
It’s on GitHub.
What’s next?
Continued development. If you have pipeline applications, I’d like to hear about them.
What if I want a …?
Just file an issue, man, that’s all I got.