How-to Guide: Capturing stdout with CWL¶

This guide explains how to capture the standard output (stdout) of a CommandLineTool in CWL.

The example focuses on the stdout and outputs blocks to save the tool's output to a file and make it available for further use.

Objective¶

Capture the stdout of a command and make it accessible as a workflow output using:

In [ ]:

cwl_file=../cwl-workflows/capture-stdout.cwl

cat $cwl_file | yq e .'$graph[1]' | tail -n 8

cwl_file=../cwl-workflows/capture-stdout.cwl cat $cwl_file | yq e .'$graph[1]' | tail -n 8

Steps¶

1. Understand the stdout Block

The stdout field specifies the file where the standard output of the CommandLineTool should be redirected. In this example:

In [ ]:

yq '.. | select(has("stdout")) | {"stdout": .stdout}'  $cwl_file

yq '.. | select(has("stdout")) | {"stdout": .stdout}' $cwl_file

2. Understand the outputs Block

The outputs block defines how to extract information from the stdout file:

In [ ]:

cat $cwl_file | yq e .'$graph[1]' | yq '.. | select(has("outputs")) | {"outputs": .outputs}' 

cat $cwl_file | yq e .'$graph[1]' | yq '.. | select(has("outputs")) | {"outputs": .outputs}'

3. Review the CWL File

Here’s the complete example (capture-stdout.cwl):

In [ ]:

yq -e . $cwl_file

yq -e . $cwl_file

It's graphical representation:

In [ ]:

cwltool --print-dot $cwl_file 2> /dev/null | dot -Tpng | display

cwltool --print-dot $cwl_file 2> /dev/null | dot -Tpng | display

4. Provide an Input

For this example, use the input file (or command-line argument) to specify the GeoTIFF:

cwltool capture-stdout.cwl \
    --tif "https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/10/T/FK/2021/7/S2B_10TFK_20210713_0_L2A/TCI.tif"

5. Execute the CWL Tool

Run the command in your terminal:

cwltool capture-stdout.cwl \
    --tif "https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/10/T/FK/2021/7/S2B_10TFK_20210713_0_L2A/TCI.tif" > output.json

In [ ]:

cat > input.json <<'JSON'
{
  "tif": {
    "value": "https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/10/T/FK/2021/7/S2B_10TFK_20210713_0_L2A/TCI.tif"
  }
}
JSON
cwltool --podman "$cwl_file" input.json > output.json

cat > input.json <<'JSON' { "tif": { "value": "https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/10/T/FK/2021/7/S2B_10TFK_20210713_0_L2A/TCI.tif" } } JSON cwltool --podman "$cwl_file" input.json > output.json

Expected Output¶

The rio info command processes the GeoTIFF file, and its output is written to the message file. The workflow then captures this file’s contents as the info output.

Sample output:

In [ ]:

cat output.json | yq e . -o=json

cat output.json | yq e . -o=json

Key Takeaways¶

• stdout Field: Redirects the standard output to a specified file.
• outputs Block:
  • Uses glob to locate the file.
  • Loads the file's contents into memory with loadContents: true.
  • Extracts and evaluates the contents with outputEval.

By focusing on the stdout and outputs blocks, this guide helps you capture and use command outputs in CWL workflows.

In [ ]:

rm -f output.json

rm -f output.json