Skip to content

Tile Based Classification

Goal

Wrap the CommandLineTool with a CWL Workflow

Lab

This step has a dedicated lab available at /workspace/inference-eoap/practice-labs/Workflow.ipynb

How to wrap a CWL CommandLineTool with a CWL Workflow

The Cloud native Workflow orchestrates the wrapped Python application command line tool as a CWL CommandLineTool step with input parameters:

  • a staged and converted to COG Sentinel-2 Level-1C acquisition
graph TB A[STAC Catalog] A --> B(("Inference")); B --> C[STAC Catalog]

The CWL Workflow is shown below and the lines highlighted chain the inference step:

tile-based-classification.cwl
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
cwlVersion: v1.0
$namespaces:
  s: https://schema.org/
s:softwareVersion: 1.0.0
schemas:
  - http://schema.org/version/9.0/schemaorg-current-http.rdf
$graph:
  - class: Workflow
    id: main
    label: Sentinel-2 Level-1C Tile Based Classification
    doc: Sentinel-2 Level-1C Tile Based Classification based on EUROSAT dataset
    requirements: {}
    inputs:
      item:
        doc: Reference to a STAC item
        label: STAC item reference
        type: Directory
    outputs:
      - id: stac_catalog
        outputSource:
          - node_inference/stac_catalog
        type: Directory
    steps:
      node_inference:
        run: "#inference"
        in:
          input-item: item
        out:
          - stac_catalog
  - class: CommandLineTool
    id: inference
    requirements:
      InlineJavascriptRequirement: {}
      EnvVarRequirement:
        envDef:
          PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
          PYTHONPATH: /app
      ResourceRequirement:
        coresMax: 2
        ramMax: 4096
    hints:
      DockerRequirement:
        dockerPull: localhost/inference:latest
    baseCommand: ["python", "-m", "app"]
    arguments: []
    inputs:
      input-item:
        type: Directory
        inputBinding:
          prefix: --input-item
    outputs:
      stac_catalog:
        outputBinding:
          glob: .
        type: Directory

To run this CWL document, one does:

terminal
export WORKSPACE=/workspace/inference-eoap

command -v podman >/dev/null 2>&1 && { 
    flag="--podman"
}

# check if the results.json file exists
if [ ! -f ${WORKSPACE}/runs/results.json ]; then
    echo "results.json file not found, run the stage-and-cog.cwl workflow first."
    exit 1
fi


cwltool ${flag} \
    --outdir ${WORKSPACE}/runs \
    ${WORKSPACE}/cwl-workflow/tile-based-classification.cwl.cwl \
    --input-item $( cat ${WORKSPACE}/runs/results.json | jq -r .stac_catalog.path )

Expected outcome

The folder /workspace/inference-eoap/runs contains: 

(base) jovyan@jupyter-fbrito--training:~/inference-eoap$ tree runs
runs
├── results.json
└── ydmns4od
    ├── S2A_T33TVM_20240125T100359_L1C
    │   ├── S2A_T33TVM_20240125T100359_L1C.json
    │   ├── blue.tif
    │   ├── cirrus.tif
    │   ├── coastal.tif
    │   ├── green.tif
    │   ├── nir.tif
    │   ├── nir08.tif
    │   ├── nir09.tif
    │   ├── red.tif
    │   ├── rededge1.tif
    │   ├── rededge2.tif
    │   ├── rededge3.tif
    │   ├── swir16.tif
    │   └── swir22.tif
    └── catalog.json

2 directories, 16 files