How-to Guide: Capturing JSON from stdout and Reusing It in Another CWL Tool¶

This guide demonstrates how to capture a JSON response from the stdout of one CWL tool and reuse it as input for another tool.

We'll focus on two key sections:

1. Capturing JSON output:

2. Reusing captured JSON in another tool:

Objective¶

• Tool 1 (stac): Fetch a STAC item in JSON format and extract its assets.
• Tool 2 (rio): Process the extracted asset to generate a PNG image based on the specified band.

Steps¶

1. Capturing JSON from stdout

In the stac tool:

Relevant Block

• stdout: message: Redirects the tool's stdout to a file named message.
• glob: message: Locates the message file.
• loadContents: true: Reads the content of the file into memory.
• outputEval: Parses the JSON content to extract the assets field using JavaScript (JSON.parse).

How It Works

• The stac tool fetches a STAC item as JSON using curl.
• The output (stdout) is saved to message.
• The outputs block processes this JSON and makes the assets field available for downstream steps.

2. Reusing Captured JSON in Another Tool

In the rio tool:

Relevant Block

• First valueFrom: Selects the asset whose eo:bands array contains a band with a common_name matching inputs.common_band_name and returns its href.
• Second valueFrom: Generates the output filename for the PNG (red.png for the red band).

How It Works

• The rio tool processes the asset URL (href) extracted in the first step.`
• It generates a PNG file using the common-band-name and saves it as `.png.

3. Complete CWL Example

Here’s the full capture-json-stdout.cwl file:

It's graphical representation:

4. Run the Example

Command to execute the workflow:

cwltool capture-json-stdout.cwl \
  --stac-item https://earth-search.aws.element84.com/v0/collections/sentinel-s2-l2a-cogs/items/S2B_53HPA_20210723_0_L2A \
  --common-band-name red

5. Execute the CWL Tool

Run the command in your terminal:

cwltool capture-json-stdout.cwl \
  --stac-item https://earth-search.aws.element84.com/v0/collections/sentinel-s2-l2a-cogs/items/S2B_53HPA_20210723_0_L2A \
  --common-band-name red

Expected Output¶

• Intermediate Output (json_output): Extracted assets from the STAC item JSON.
• Final Output (preview): PNG file for the specified band (e.g., red.png).

Key Takeaways¶

• Capturing JSON:

  • Use stdout to redirect the JSON output to a file.
  • Parse and extract specific fields using outputEval.

• Reusing JSON:

  • Use valueFrom in arguments to select fields dynamically from the JSON input.
  • Process the selected field (e.g., generate a file based on its href).

By focusing on stdout and outputBinding, this guide highlights how CWL facilitates JSON data flow between tools.