Skip to content

Crop

Goal

Wrap the crop step as a Common Workflow Language CommandLineTool and execute it with a CWL runner.

Lab

This step has a dedicated lab at practice-labs/3-CWL-CommandLineTools/crop.ipynb.

How to wrap a step as a CWL CommandLineTool

The CWL document below shows the crop step wrapped as a CWL CommandLineTool:

cwl-cli/crop.cwl
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
cwlVersion: v1.2
$namespaces:
  s: https://schema.org/
# Training-example authors and creation date mirror codemeta.json.
s:name: Crop a STAC item band
s:description: Crop a selected raster band to an area of interest.
s:dateCreated: '2022-09-01'
s:license:
  s:name: Creative Commons Attribution-ShareAlike 4.0 International
  s:url: https://creativecommons.org/licenses/by-sa/4.0/
  s:identifier: CC-BY-SA-4.0
s:softwareHelp:
  s:name: Mastering Earth Observation Application Packaging with CWL
  s:url: https://eoap.github.io/mastering-app-package
s:publisher:
  s:name: EO Application Packaging
  s:identifier: https://github.com/eoap
s:author:
  - s:givenName: Jane
    s:familyName: Doe
    s:email: jane.doe@acme.earth
    s:affiliation:
      s:name: ACME
  - s:givenName: John
    s:familyName: Doe
    s:email: john.doe@acme.earth
    s:affiliation:
      s:name: ACME
s:softwareRequirements:
  - container runtime
  - cwl runner
s:softwareVersion: 2.0.2

class: CommandLineTool
id: crop
requirements:
  InlineJavascriptRequirement: {}
  ResourceRequirement:
    coresMax: 1
    ramMax: 512
  NetworkAccess:
    networkAccess: true
  DockerRequirement:
    dockerPull: localhost/crop:latest
  SchemaDefRequirement:
    types:
      - $import: https://raw.githubusercontent.com/eoap/schemas/main/geojson.yaml
      - $import: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml
baseCommand: crop
arguments: []
inputs:
  item:
    type: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI
    inputBinding:
      prefix: --input-item
      valueFrom: $(self.value)
    label: STAC item
    doc: STAC item reference identifying the source acquisition.
  aoi:
    type: https://raw.githubusercontent.com/eoap/schemas/main/geojson.yaml#Polygon
    inputBinding:
      prefix: --aoi
      valueFrom: $(JSON.stringify(self))
    label: Area of interest
    doc: GeoJSON Polygon defining the area to crop; raster pixels outside the polygon are masked.
  epsg:
    type: string
    inputBinding:
      prefix: --epsg
      valueFrom: $(self.split(":").pop())
    label: Coordinate reference system
    doc: EPSG code of the coordinate reference system used for the area of interest.
  band:
    type: string
    inputBinding:
      prefix: --band
    label: Raster band
    doc: Name of the STAC asset band to crop.
outputs:
  cropped:
    outputBinding:
      glob: '*.tif'
    type: File



    label: Cropped raster
    doc: Raster band cropped to the area of interest.

Let's break down the key components of this CWL document:

  • cwlVersion: v1.2: Specifies the version of the CWL specification that this document follows.
  • class: CommandLineTool: Indicates that this CWL document defines a command-line tool.
  • id: crop: Provides a unique identifier for this tool, which can be used to reference it in workflows.
  • requirements: Specifies the requirements and dependencies of the tool. In this case, it defines the following:
    • InlineJavascriptRequirement: This requirement allows the use of inline JavaScript expressions in the tool.
    • EnvVarRequirement: It sets environment variables. In this case, it adds /app/venv/bin to PATH so the installed commands can be found.
    • ResourceRequirement: Specifies resource requirements for running the tool, including the maximum number of CPU cores and maximum RAM.
    • DockerRequirement: This requirement specifies the Docker container to be used. It indicates that the tool should be executed in a Docker container with the image localhost/crop:latest.
  • baseCommand: Defines the base command to be executed in the container. In this case, it runs the installed crop command.
  • arguments: This section is empty, meaning there are no additional command-line arguments specified here. The tool is expected to receive its arguments via the input parameters.
  • inputs: Describes the input parameters for the tool, including their types and how they are bound to command-line arguments. The tool expects the following inputs:
    • item: A URI record containing a value field with the STAC item URL; valueFrom: $(self.value) passes that URL, bound to the --input-item argument.
    • aoi: A GeoJSON Polygon record with coordinates and bbox; valueFrom: $(JSON.stringify(self)) serializes the full Polygon, bound to the --aoi argument.
    • epsg: A string representing the EPSG code for the coordinate system, bound to the --epsg argument.
    • band: A string representing the name of the band to be extracted, bound to the --band argument.
  • outputs: Specifies the tool's output. It defines an output parameter named cropped, which is of type File. The outputBinding section specifies that the tool is expected to produce one or more TIFF files (glob: '*.tif') as output.

Steps

Clean-up the /workspace/mastering-app-package/runs folder: 

rm -fr /workspace/mastering-app-package/runs/*

Run the CWL document using the cwltool CWL runner to execute the crop step with the green band with:

terminal
export WORKSPACE=${WORKSPACE:-/workspace/mastering-app-package}
mkdir -p "${WORKSPACE}/runs"

cwltool \
    --podman \
    --outdir "${WORKSPACE}/runs" \
    "${WORKSPACE}/cwl-cli/crop.cwl" \
    "${WORKSPACE}/cwl-cli/crop-green-params.yaml"

sh -x ${WORKSPACE}/scripts/cwl-cli-crop-green.sh

Run the CWL document using the cwltool CWL runner to execute the crop step with the nir band with:

terminal
export WORKSPACE=${WORKSPACE:-/workspace/mastering-app-package}
mkdir -p "${WORKSPACE}/runs"

cwltool \
    --podman \
    --outdir "${WORKSPACE}/runs" \
    "${WORKSPACE}/cwl-cli/crop.cwl" \
    "${WORKSPACE}/cwl-cli/crop-params.yaml"

sh -x ${WORKSPACE}/scripts/cwl-cli-crop-nir.sh

Expected outcome

The folder /workspace/mastering-app-package/runs contains: 

(base) jovyan@coder-mrossi:~/runs$ tree .
.
├── crop_green.tif
└── crop_nir.tif

0 directories, 2 files

Extra

The CWL runner cwltool allows you to do a YAML file with the parameters:

crop-params.yaml
item:
  value: https://earth-search.aws.element84.com/v0/collections/sentinel-s2-l2a-cogs/items/S2B_10TFK_20210713_0_L2A
aoi:
  type: Polygon
  coordinates:
    -   -   - -121.399
            - 39.834
        -   - -120.74
            - 39.834
        -   - -120.74
            - 40.472
        -   - -121.399
            - 40.472
        -   - -121.399
            - 39.834
  bbox:
    - -121.399
    - 39.834
    - -120.74
    - 40.472
epsg: EPSG:4326
band: nir

and run it with:

terminal
cwltool \
    --podman \
    --outdir /workspace/mastering-app-package/runs \
    ${WORKSPACE}/cwl-cli/crop.cwl \
    ${WORKSPACE}/cwl-cli/crop-params.yaml

SchemaDefRequirement imports the EOAP GeoJSON and URI record schemas. Use the YAML job above to supply structured inputs. The standalone crop tool retains string epsg and band inputs; the workflows constrain these through enums. The binding passes the full Polygon as JSON, and the application masks pixels outside it.