Pipeline

A Pipeline comprises stages that can be combined to manipulate data from any source before being written to target(s).

When data is written to the Pipeline it’s called an event. Events propagate through the pipeline carrying a value (ex. source value) and metadata (ex. source name). The event flows through the pipeline given the stage layout, being processed by each stage, before eventually being written to one or more targets.

Pipelines receive events from Flows. To send an event to a Pipeline, configure a Flow Target to the Pipeline.

Creating a pipeline

1. Click Pipeline in the configuration’s Main Menu and then click New Pipeline to get started.

2. Enter a Name to represent the pipeline. Names can only contain alphanumeric and underscore characters (e.g., A-Z, a-z, 0-9 or _). Optionally enter in a Description and configure Tags. Click *Next to continue.

3. Use the Add Stage menu to create 1..N Stages. At least one stage is required to continue.

4. Use the Stage Layout builder to connect stage outputs (bottom) to stage inputs (top) as desired.

5. Choose submit to create the Pipeline. Once created, use the Pipeline as a Flow target to send it data.

Learn more about pipelines:

• Stages
• Tools