Skip to content

Pattern 10

2025-09-08 10:55:33.192 | DEBUG    | cwl_loader:load_cwl_from_location:196 - Loading CWL document from ../cwl-workflow/pattern-10.cwl...

2025-09-08 10:55:33.192 | DEBUG    | cwl_loader:_load_cwl_from_stream:199 - Reading stream from ../cwl-workflow/pattern-10.cwl...

2025-09-08 10:55:33.224 | DEBUG    | cwl_loader:load_cwl_from_stream:173 - CWL data of type <class 'ruamel.yaml.comments.CommentedMap'> successfully loaded from stream

2025-09-08 10:55:33.224 | DEBUG    | cwl_loader:load_cwl_from_yaml:124 - Updating the model of type 'CommentedMap' to version 'v1.2'...

2025-09-08 10:55:33.229 | DEBUG    | cwl_loader:load_cwl_from_yaml:135 - Raw CWL document successfully updated to v1.2! Now converting to the CWL model...

2025-09-08 10:55:33.241 | DEBUG    | cwl_loader:load_cwl_from_yaml:143 - Raw CWL document successfully updated to v1.2! Now dereferencing the FQNs...

2025-09-08 10:55:33.241 | DEBUG    | cwl_loader:_clean_process:66 -   Cleaning Workflow pattern-10...

2025-09-08 10:55:33.242 | DEBUG    | cwl_loader:_clean_process:66 -   Cleaning Workflow vegetation_indexes...

2025-09-08 10:55:33.242 | DEBUG    | cwl_loader:_clean_process:66 -   Cleaning CommandLineTool clt...

2025-09-08 10:55:33.243 | DEBUG    | cwl_loader:_clean_process:66 -   Cleaning ExpressionTool flatten_array...

2025-09-08 10:55:33.243 | DEBUG    | cwl_loader:load_cwl_from_yaml:151 - CWL document successfully dereferenced!

2025-09-08 10:55:33.243 | DEBUG    | cwl_loader:_load_cwl_from_stream:207 - Stream from ../cwl-workflow/pattern-10.cwl successfully load!

Application Package Pattern 10¶

This pattern shows how to flatten an array of array of Directory

No description has been provided for this image

Inputs¶

Id	Type	Label	Doc
`aoi`	`string`	area of interest	area of interest as a bounding box
`epsg`	`string`	EPSG code	EPSG code
`indexes`	`string[]`	indexes	indexes to compute
`items`	`Directory[]`	Landsat-8/9 acquisition reference	Landsat-8/9 acquisition reference

Steps¶

Id	Runs	Label	Doc
`subworkflow`	`#vegetation_indexes`	None	None
`flatten`	`#flatten_array`	None	None

Outputs¶

Id	Type	Label	Doc
`vegetation_indexes`	`Directory[]`	Vegetation indexes	Vegetation indexes from Landsat-8/9 acquisitions

Data flow management¶

2025-09-08 10:55:34.140 | DEBUG    | cwl_loader:load_cwl_from_location:196 - Loading CWL document from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/develop/templates/stage-in.cwl...

2025-09-08 10:55:34.291 | DEBUG    | cwl_loader:_load_cwl_from_stream:199 - Reading stream from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/develop/templates/stage-in.cwl...

2025-09-08 10:55:34.298 | DEBUG    | cwl_loader:load_cwl_from_stream:173 - CWL data of type <class 'ruamel.yaml.comments.CommentedMap'> successfully loaded from stream

2025-09-08 10:55:34.298 | DEBUG    | cwl_loader:load_cwl_from_yaml:124 - Updating the model of type 'CommentedMap' to version 'v1.2'...

2025-09-08 10:55:34.299 | DEBUG    | cwl_loader:load_cwl_from_yaml:135 - Raw CWL document successfully updated to v1.2! Now converting to the CWL model...

2025-09-08 10:55:34.454 | DEBUG    | cwl_loader:load_cwl_from_yaml:143 - Raw CWL document successfully updated to v1.2! Now dereferencing the FQNs...

2025-09-08 10:55:34.454 | DEBUG    | cwl_loader:_clean_process:66 -   Cleaning CommandLineTool my-asthonishing-stage-in-directory...

2025-09-08 10:55:34.455 | DEBUG    | cwl_loader:load_cwl_from_yaml:151 - CWL document successfully dereferenced!

2025-09-08 10:55:34.455 | DEBUG    | cwl_loader:_load_cwl_from_stream:207 - Stream from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/develop/templates/stage-in.cwl successfully load!

2025-09-08 10:55:34.456 | DEBUG    | cwl_loader:load_cwl_from_location:196 - Loading CWL document from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/develop/templates/stage-in-file.cwl...

2025-09-08 10:55:34.489 | DEBUG    | cwl_loader:_load_cwl_from_stream:199 - Reading stream from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/develop/templates/stage-in-file.cwl...

2025-09-08 10:55:34.495 | DEBUG    | cwl_loader:load_cwl_from_stream:173 - CWL data of type <class 'ruamel.yaml.comments.CommentedMap'> successfully loaded from stream

2025-09-08 10:55:34.495 | DEBUG    | cwl_loader:load_cwl_from_yaml:124 - Updating the model of type 'CommentedMap' to version 'v1.2'...

2025-09-08 10:55:34.496 | DEBUG    | cwl_loader:load_cwl_from_yaml:135 - Raw CWL document successfully updated to v1.2! Now converting to the CWL model...

2025-09-08 10:55:34.638 | DEBUG    | cwl_loader:load_cwl_from_yaml:143 - Raw CWL document successfully updated to v1.2! Now dereferencing the FQNs...

2025-09-08 10:55:34.639 | DEBUG    | cwl_loader:_clean_process:66 -   Cleaning CommandLineTool my-asthonishing-stage-in-file...

2025-09-08 10:55:34.639 | DEBUG    | cwl_loader:load_cwl_from_yaml:151 - CWL document successfully dereferenced!

2025-09-08 10:55:34.640 | DEBUG    | cwl_loader:_load_cwl_from_stream:207 - Stream from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/develop/templates/stage-in-file.cwl successfully load!

2025-09-08 10:55:34.640 | DEBUG    | cwl_loader:load_cwl_from_location:196 - Loading CWL document from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/develop/templates/stage-out.cwl...

2025-09-08 10:55:34.783 | DEBUG    | cwl_loader:_load_cwl_from_stream:199 - Reading stream from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/develop/templates/stage-out.cwl...

2025-09-08 10:55:34.793 | DEBUG    | cwl_loader:load_cwl_from_stream:173 - CWL data of type <class 'ruamel.yaml.comments.CommentedMap'> successfully loaded from stream

2025-09-08 10:55:34.793 | DEBUG    | cwl_loader:load_cwl_from_yaml:124 - Updating the model of type 'CommentedMap' to version 'v1.2'...

2025-09-08 10:55:34.794 | DEBUG    | cwl_loader:load_cwl_from_yaml:135 - Raw CWL document successfully updated to v1.2! Now converting to the CWL model...

2025-09-08 10:55:34.944 | DEBUG    | cwl_loader:load_cwl_from_yaml:143 - Raw CWL document successfully updated to v1.2! Now dereferencing the FQNs...

2025-09-08 10:55:34.945 | DEBUG    | cwl_loader:_clean_process:66 -   Cleaning CommandLineTool my-super-stage-out...

2025-09-08 10:55:34.945 | DEBUG    | cwl_loader:load_cwl_from_yaml:151 - CWL document successfully dereferenced!

2025-09-08 10:55:34.946 | DEBUG    | cwl_loader:_load_cwl_from_stream:207 - Stream from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/develop/templates/stage-out.cwl successfully load!

2025-09-08 10:55:34.947 | INFO     | eoap_cwlwrap.types:_validate_stage_in:273 - Validating stage-in 'my-asthonishing-stage-in-directory'...

2025-09-08 10:55:34.947 | INFO     | eoap_cwlwrap.types:_validate_stage_in:295 - stage-in 'my-asthonishing-stage-in-directory' is valid

2025-09-08 10:55:34.948 | INFO     | eoap_cwlwrap.types:_validate_stage_in:273 - Validating stage-in 'my-asthonishing-stage-in-file'...

2025-09-08 10:55:34.949 | INFO     | eoap_cwlwrap.types:_validate_stage_in:295 - stage-in 'my-asthonishing-stage-in-file' is valid

2025-09-08 10:55:34.949 | INFO     | eoap_cwlwrap.types:validate_stage_out:331 - Validating stage-out 'my-super-stage-out'...

2025-09-08 10:55:34.950 | INFO     | eoap_cwlwrap.types:validate_stage_out:353 - stage-out 'my-super-stage-out' is valid

2025-09-08 10:55:34.950 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:90 - Building the CWL Orchestrator Workflow...

2025-09-08 10:55:34.951 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:119 - Analyzing pattern-10 inputs...

2025-09-08 10:55:34.951 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:135 - * pattern-10/aoi: string

2025-09-08 10:55:34.952 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:135 - * pattern-10/epsg: string

2025-09-08 10:55:34.953 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:135 - * pattern-10/indexes: string[]

2025-09-08 10:55:34.953 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:135 - * pattern-10/items: Directory[]

2025-09-08 10:55:34.954 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:148 -   Directory type detected, creating a related 'directory_stage_in_0'...

2025-09-08 10:55:34.956 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:150 -   Converting Directory[] to URL-compatible type...

2025-09-08 10:55:34.957 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:154 -   Directory[] converted to https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI[]

2025-09-08 10:55:34.957 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:175 -   Array detected, 'scatter' required for reference:items

2025-09-08 10:55:34.958 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:195 -   Connecting 'app/items' to 'directory_stage_in_0' output...

2025-09-08 10:55:34.959 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:246 - Analyzing pattern-10 outputs...

2025-09-08 10:55:34.959 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:252 - * pattern-10/vegetation_indexes: Directory[]

2025-09-08 10:55:34.959 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:257 -   Directory type detected, creating a related 'stage_out_0'...

2025-09-08 10:55:34.960 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:259 -   Converting Directory[] to URL-compatible type...

2025-09-08 10:55:34.960 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:263 -   Directory[] converted to https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI[]

2025-09-08 10:55:34.962 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:284 -   Array detected, scatter required for stac_catalog:app/vegetation_indexes

2025-09-08 10:55:34.962 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:304 -   Connecting 'app/vegetation_indexes' to 'stage_out_0' output...

2025-09-08 10:55:34.963 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:379 - Orchestrator Workflow built in 0.0127 seconds

No description has been provided for this image

Workflow components diagram¶

No description has been provided for this image

Inputs¶

Id	Type	Label	Doc
`aoi`	`string`	area of interest - pattern-10/aoi	area of interest as a bounding box - This parameter is derived from pattern-10/aoi
`epsg`	`string`	EPSG code - pattern-10/epsg	EPSG code - This parameter is derived from pattern-10/epsg
`indexes`	`string[]`	indexes - pattern-10/indexes	indexes to compute - This parameter is derived from pattern-10/indexes
`another_input`	`string`	Another Input - my-asthonishing-stage-in-directory/another_input	An additional input for demonstration purposes - This parameter is derived from my-asthonishing-stage-in-directory/another_input
`items`	`https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI[]`	Landsat-8/9 acquisition reference - pattern-10/items	Landsat-8/9 acquisition reference - This parameter is derived from pattern-10/items
`s3_bucket`	`string`	my-super-stage-out/s3_bucket	This parameter is derived from: my-super-stage-out/s3_bucket
`sub_path`	`string`	my-super-stage-out/sub_path	This parameter is derived from: my-super-stage-out/sub_path
`aws_access_key_id`	`string`	my-super-stage-out/aws_access_key_id	This parameter is derived from: my-super-stage-out/aws_access_key_id
`aws_secret_access_key`	`string`	my-super-stage-out/aws_secret_access_key	This parameter is derived from: my-super-stage-out/aws_secret_access_key
`region_name`	`string`	my-super-stage-out/region_name	This parameter is derived from: my-super-stage-out/region_name
`endpoint_url`	`string`	my-super-stage-out/endpoint_url	This parameter is derived from: my-super-stage-out/endpoint_url

Steps¶

Id	Runs	Label	Doc
`directory_stage_in_0`	`#my-asthonishing-stage-in-directory`	None	None
`app`	`#pattern-10`	None	None
`stage_out_0`	`#my-super-stage-out`	None	None

Outputs¶

Id	Type	Label	Doc
`vegetation_indexes`	`https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI[]`	Vegetation indexes	Vegetation indexes from Landsat-8/9 acquisitions

Execution¶

Inputs¶

{
  "aoi": "-118.985,38.432,-118.183,38.938",
  "epsg": "EPSG:4326",
  "indexes": [
    "ndvi",
    "ndwi"
  ],
  "items": [
    {
      "class": "https://raw.githubusercontent.com/eoap/schemas/main/url.yaml#URL",
      "value": "https://planetarycomputer.microsoft.com/api/stac/v1/collections/landsat-c2-l2/items/LC08_L2SP_042033_20231007_02_T1"
    },
    {
      "class": "https://raw.githubusercontent.com/eoap/schemas/main/url.yaml#URL",
      "value": "https://planetarycomputer.microsoft.com/api/stac/v1/collections/landsat-c2-l2/items/LC09_L2SP_042033_20231015_02_T1"
    }
  ]
}

Outputs¶

{
    "vegetation_indexes": [
        {
            "value": "s3://my-bucket/my/sub/path-hg1i9n/catalog.json",
            "type": "https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI"
        },
        {
            "value": "s3://my-bucket/my/sub/path-1aydsb/catalog.json",
            "type": "https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI"
        },
        {
            "value": "s3://my-bucket/my/sub/path-bjotbu/catalog.json",
            "type": "https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI"
        },
        {
            "value": "s3://my-bucket/my/sub/path-xmhw4g/catalog.json",
            "type": "https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI"
        }
    ]
}