How-to Guide: Capturing stdout with CWL¶

This guide explains how to capture the standard output (stdout) of a CommandLineTool in CWL.

The example focuses on the stdout and outputs blocks to save the tool's output to a file and make it available for further use.

outputs:

  preview:

    type: File

    outputBinding:

      glob: preview.png

This block ensures the tool captures and returns the preview.png file generated during execution.

Objective¶

Convert a Sentinel-2 GeoTIFF file into a PNG image.

Capture the generated PNG file as the tool's output using the outputs block.

Steps¶

Understand the outputs Block

In CWL, the outputs block specifies:

Name: preview – the logical name of the output.
Type: File – indicates the output is a file.
outputBinding.glob: Specifies the file's name or pattern to locate the file in the tool's working directory.

From the example:

outputs:

  preview:

    type: File

    outputBinding:

      glob: preview.png

glob: preview.png: Ensures the tool captures the file preview.png generated by the rio convert command.

Review the return-output-file.cwl Workflow

The workflow (return-output-file.cwl) is defined as:

cwlVersion: v1.2

$graph:

  - class: Workflow

    id: main

    inputs:

      tif:

        type: string

        label: URL to a Sentinel-2 TCI GeoTIFF

        doc: URL to a Sentinel-2 True Colour Image GeoTIFF file (TCI.tif)

    outputs:

      preview:

        outputSource: step-convert/preview

        type: File

        label: True Colour Image preview

        doc: True Colour Image preview in PNG format

    steps:

      step-convert:

in:

          geotif: tif

        out:

          - preview

        run: "#rio"

  - class: CommandLineTool

    id: rio

    label: Rasterio command line tool

    doc: Convert a GeoTIFF file to a PNG file using rio convert

    requirements:

      NetworkAccess:

        networkAccess: true

      InlineJavascriptRequirement: {}

      DockerRequirement:

        dockerPull: ghcr.io/eoap/how-to/rio:1.0.0

    baseCommand: rio

    arguments:

      - convert

      - --driver

      - PNG

      - --dtype

      - uint8

      - $(inputs.geotif)

      - preview.png

    inputs:

      geotif:

        type: string

    outputs:

      preview:

        type: File

        outputBinding:

          glob: preview.png

It's graphical representation:

No description has been provided for this image

Provide Input Parameters

Create an input file inputs.yml with the GeoTIFF URL:

tif: "https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/10/T/FK/2021/7/S2B_10TFK_20210713_0_L2A/TCI.tif"

Execute the Workflow

Run the workflow using cwltool:

cwltool return-output-file input.yml

INFO /opt/hostedtoolcache/Python/3.12.8/x64/bin/cwltool 3.1.20241217163858

INFO Resolved '../cwl-workflows/return-output-file.cwl' to 'file:///home/runner/work/how-to/how-to/cwl-workflows/return-output-file.cwl'

INFO [workflow ] start

INFO [workflow ] starting step step-convert

INFO [step step-convert] start

INFO [job step-convert] /tmp/kcyf0xlh$ docker \

    run \

    -i \

    --mount=type=bind,source=/tmp/kcyf0xlh,target=/nYHzrL \

    --mount=type=bind,source=/tmp/kdx3wpg3,target=/tmp \

    --workdir=/nYHzrL \

    --read-only=true \

    --user=1001:128 \

    --rm \

    --cidfile=/tmp/89gxdz2l/20250102120352-836026.cid \

    --env=TMPDIR=/tmp \

    --env=HOME=/nYHzrL \

    ghcr.io/eoap/how-to/rio:1.0.0 \

    rio \

    convert \

    --driver \

    PNG \

    --dtype \

    uint8 \

    https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/10/T/FK/2021/7/S2B_10TFK_20210713_0_L2A/TCI.tif \

    preview.png

WARNING:rasterio._env:CPLE_NotSupported in driver PNG does not support creation option BLOCKXSIZE

WARNING:rasterio._env:CPLE_NotSupported in driver PNG does not support creation option BLOCKYSIZE

WARNING:rasterio._env:CPLE_NotSupported in driver PNG does not support creation option TILED

WARNING:rasterio._env:CPLE_NotSupported in driver PNG does not support creation option COMPRESS

WARNING:rasterio._env:CPLE_NotSupported in driver PNG does not support creation option INTERLEAVE

INFO [job step-convert] Max memory used: 1137MiB

INFO [job step-convert] completed success

INFO [step step-convert] completed success

INFO [workflow ] completed success

INFO Final process status is success

Expected Output¶

The rio tool processes the input GeoTIFF and generates a PNG file named preview.png.

The outputs block ensures:

The file is located using glob: preview.png.
It is made available as the workflow output.

Example Workflow Output:

{"preview": {"location": "file:///home/runner/work/how-to/how-to/docs/preview.png", "basename": "preview.png", "class": "File", "checksum": "sha1$e1538ef65976d4748dc58393fe8d0b78b4736f38", "size": 237388032, "path": "/home/runner/work/how-to/how-to/docs/preview.png"}}

The preview.png file is now accessible in the working directory or as defined by the CWL runner.

Key Takeaways¶

The outputs block defines how to capture and return files:
- type: File: Specifies the output type.
- outputBinding.glob: Matches the file name or pattern.
Ensure the file name in glob matches the file name generated by the tool (preview.png).

By focusing on the outputs block, this guide highlights how to capture and return output files in CWL workflows.