How-to Guide: Scattering on Input Parameters¶

This guide explains how to scatter workflow steps based on input parameters using ScatterFeatureRequirement and MultipleInputFeatureRequirement.

The focus is on creating workflows where multiple input parameters are processed in parallel.

Objective¶

Scatter the step_curl task to process multiple input bands (red, green, blue) in parallel and combine their results in subsequent steps.

Key Features¶

1. ScatterFeatureRequirement

Enables scattering, allowing parallel execution of workflow steps for array-like inputs.

2. MultipleInputFeatureRequirement

Allows using multiple input fields in a workflow.

3. Scatter on common_band_name

Scatters the step_curl task based on the common_band_name parameter array.

Steps¶

1. Define the Workflow

The workflow uses default input values for the band names (red, green, blue) and processes them using scattering.

Workflow Inputs

The workflow accepts:

• stac-item: URL of a STAC item.
• red-band: Band name for the red channel (default: "red").
• green-band: Band name for the green channel (default: "green").
• blue-band: Band name for the blue channel (default: "blue").

Workflow Outputs

• The final output is the RGB composite TIFF file (rgb-tif).

Workflow Definition

Its graphical representation:

2. Scatter Configuration

• Step 1: Scatter on common_band_name

The step_curl step scatters over the common_band_name parameter array ([red-band, green-band, blue-band]).

• scatter: Scatters the step over the values of common_band_name.
• scatterMethod: dotproduct: Matches corresponding elements in the array for parallel execution.

Step 2: Combine Results

The step_stack step combines the TIFF files fetched by step_curl.

3. Substeps and Tools

stac Tool

• Fetches URLs of the TIFF files corresponding to each band.

rio_stack Tool

• Stacks the TIFF files into a single composite file.

4. Run the Workflow

Execute the workflow with the default parameters:

cwltool scatter-input-parameters.cwl \
    --stac-item https://earth-search.aws.element84.com/v0/collections/sentinel-s2-l2a-cogs/items/S2B_53HPA_20210723_0_L2A

5. Expected Output

Intermediate Outputs:

• hrefs: URLs of the TIFF files for red, green, and blue bands.
• stacked.tif: Composite TIFF file of all bands.

Final Output:

• rgb-tif: RGB composite TIFF file.

Key Takeaways¶

Scattering with Input Parameters:

• The scatter field enables parallel execution over array inputs.
• scatterMethod: dotproduct ensures corresponding elements in arrays are processed together.

Multiple Input Fields:

• MultipleInputFeatureRequirement allows combining multiple input parameters in a single scatter operation.

Parallel and Modular Design:

• Scattering simplifies workflows by enabling parallel processing of input parameters.

This guide demonstrates how to use scattering to process multiple input parameters in parallel within a CWL workflow.