Map units

Learn how to use map units to work with collections in data sources.

Map units allow you to process data collections in sub-pipelines and then provide the results back to the main pipeline for further processing. Each element in a collection is processed in its own pipeline. You build a sub-pipeline as is done for pipelines.

Video overview

View this short video for a brief overview of the Map unit.

Map unit overview

Collection data sources

To use the Map unit, you need a data source containing at least one collection. If an attribute is a collection, it is indicated by a Collections icon, as shown in the figure below.

Input collection for the Map unit

Each collection element processed in its own sub-pipeline

When the data flow enters the Map unit, the flow iterates through each element in the collection. For example, if the collection has two elements, then the map unit does a single call; if the collection has three elements, then the map unit does three calls; and so on. You can add additional data sources to the sub-pipeline, as required.

The results of each execution are then merged into a single output and provided to the main pipeline through the Results unit.

Example sub-pipeline within a Map unit
  1. In a pipeline, drag a line to a unit and drop it onto a connection indicator.

  2. From the Select unit to place pane, click Map Unit to add it to the pipeline.

  3. In the Edit Map area, select a collection from the data source and drop it into Input Collection field.

  4. If required, add Context attributes within the parameters folder. You can drag them from the data source to the parameters field.

  5. Build the sub-pipeline.

  6. Click Inspect Sub Pipeline to view the sub-pipeline. You can add data sources, map attributes to parameters and so on.

  7. Click Done to save changes.

Map unit added to pipeline
Adding nest (collection) to Map unit
Map unit completed
Map unit added to pipeline