Skip to content

Pattern 10

2025-12-02 12:42:45.960 | DEBUG    | cwl_loader:load_cwl_from_location:228 - Loading CWL document from ../cwl-workflow/pattern-10.cwl...

2025-12-02 12:42:45.960 | DEBUG    | cwl_loader:_load_cwl_from_stream:231 - Reading stream from ../cwl-workflow/pattern-10.cwl...

2025-12-02 12:42:45.998 | DEBUG    | cwl_loader:load_cwl_from_stream:203 - CWL data of type <class 'ruamel.yaml.comments.CommentedMap'> successfully loaded from stream

2025-12-02 12:42:45.998 | DEBUG    | cwl_loader:load_cwl_from_yaml:130 - Updating the model from version 'v1.0' to version 'v1.2'...

2025-12-02 12:42:45.999 | DEBUG    | cwl_loader:load_cwl_from_yaml:141 - Raw CWL document successfully updated to v1.2!

2025-12-02 12:42:45.999 | DEBUG    | cwl_loader:load_cwl_from_yaml:145 - Parsing the raw CWL document to the CWL Utils DOM...

2025-12-02 12:42:46.079 | DEBUG    | cwl_loader:load_cwl_from_yaml:158 - Raw CWL document successfully parsed to the CWL Utils DOM!

2025-12-02 12:42:46.079 | DEBUG    | cwl_loader:load_cwl_from_yaml:160 - Dereferencing the steps[].run...

2025-12-02 12:42:46.080 | DEBUG    | cwl_loader:_on_process:78 - Checking if ../cwl-workflow/pattern-10.cwl#vegetation_indexes must be externally imported...

2025-12-02 12:42:46.080 | DEBUG    | cwl_loader:_on_process:82 - run_url: ../cwl-workflow/pattern-10.cwl - uri: ../cwl-workflow/pattern-10.cwl

2025-12-02 12:42:46.081 | DEBUG    | cwl_loader:_on_process:78 - Checking if ../cwl-workflow/pattern-10.cwl#flatten_array must be externally imported...

2025-12-02 12:42:46.081 | DEBUG    | cwl_loader:_on_process:82 - run_url: ../cwl-workflow/pattern-10.cwl - uri: ../cwl-workflow/pattern-10.cwl

2025-12-02 12:42:46.082 | DEBUG    | cwl_loader:_on_process:78 - Checking if ../cwl-workflow/pattern-10.cwl#clt must be externally imported...

2025-12-02 12:42:46.082 | DEBUG    | cwl_loader:_on_process:82 - run_url: ../cwl-workflow/pattern-10.cwl - uri: ../cwl-workflow/pattern-10.cwl

2025-12-02 12:42:46.082 | DEBUG    | cwl_loader:load_cwl_from_yaml:167 - steps[].run successfully dereferenced! Dereferencing the FQNs...

2025-12-02 12:42:46.084 | DEBUG    | cwl_loader:load_cwl_from_yaml:171 - CWL document successfully dereferenced! Now verifying steps[].run integrity...

2025-12-02 12:42:46.084 | DEBUG    | cwl_loader:load_cwl_from_yaml:175 - All steps[].run link are resolvable!

2025-12-02 12:42:46.085 | DEBUG    | cwl_loader:load_cwl_from_yaml:178 - Sorting Process instances by dependencies....

2025-12-02 12:42:46.085 | DEBUG    | cwl_loader:load_cwl_from_yaml:180 - Sorting process is over.

2025-12-02 12:42:46.085 | DEBUG    | cwl_loader:_load_cwl_from_stream:240 - Stream from ../cwl-workflow/pattern-10.cwl successfully load!

Application Package Pattern 10¶

This pattern shows how to flatten an array of array of Directory

No description has been provided for this image

Inputs¶

Id	Type	Label	Doc
`aoi`	`string`	area of interest	area of interest as a bounding box
`epsg`	`string`	EPSG code	EPSG code
`indexes`	`string[]`	indexes	indexes to compute
`items`	`Directory[]`	Landsat-8/9 acquisition reference	Landsat-8/9 acquisition reference

Steps¶

Id	Runs	Label	Doc
`subworkflow`	`#vegetation_indexes`	None	None
`flatten`	`#flatten_array`	None	None

Outputs¶

Id	Type	Label	Doc
`vegetation_indexes`	`Directory[]`	Vegetation indexes	Vegetation indexes from Landsat-8/9 acquisitions

Data flow management¶

2025-12-02 12:42:46.967 | DEBUG    | cwl_loader:load_cwl_from_location:228 - Loading CWL document from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/develop/templates/stage-in.cwl...

2025-12-02 12:42:46.999 | DEBUG    | cwl_loader:_load_cwl_from_stream:231 - Reading stream from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/develop/templates/stage-in.cwl...

2025-12-02 12:42:47.006 | DEBUG    | cwl_loader:load_cwl_from_stream:203 - CWL data of type <class 'ruamel.yaml.comments.CommentedMap'> successfully loaded from stream

2025-12-02 12:42:47.006 | DEBUG    | cwl_loader:load_cwl_from_yaml:130 - Updating the model from version 'v1.0' to version 'v1.2'...

2025-12-02 12:42:47.007 | DEBUG    | cwl_loader:load_cwl_from_yaml:141 - Raw CWL document successfully updated to v1.2!

2025-12-02 12:42:47.008 | DEBUG    | cwl_loader:load_cwl_from_yaml:145 - Parsing the raw CWL document to the CWL Utils DOM...

2025-12-02 12:42:47.186 | DEBUG    | cwl_loader:load_cwl_from_yaml:158 - Raw CWL document successfully parsed to the CWL Utils DOM!

2025-12-02 12:42:47.186 | DEBUG    | cwl_loader:load_cwl_from_yaml:160 - Dereferencing the steps[].run...

2025-12-02 12:42:47.187 | DEBUG    | cwl_loader:load_cwl_from_yaml:167 - steps[].run successfully dereferenced! Dereferencing the FQNs...

2025-12-02 12:42:47.187 | DEBUG    | cwl_loader:load_cwl_from_yaml:171 - CWL document successfully dereferenced! Now verifying steps[].run integrity...

2025-12-02 12:42:47.188 | DEBUG    | cwl_loader:load_cwl_from_yaml:175 - All steps[].run link are resolvable!

2025-12-02 12:42:47.189 | DEBUG    | cwl_loader:load_cwl_from_yaml:178 - Sorting Process instances by dependencies....

2025-12-02 12:42:47.190 | DEBUG    | cwl_loader:load_cwl_from_yaml:180 - Sorting process is over.

2025-12-02 12:42:47.190 | DEBUG    | cwl_loader:_load_cwl_from_stream:240 - Stream from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/develop/templates/stage-in.cwl successfully load!

2025-12-02 12:42:47.191 | DEBUG    | cwl_loader:load_cwl_from_location:228 - Loading CWL document from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/develop/templates/stage-in-file.cwl...

2025-12-02 12:42:47.231 | DEBUG    | cwl_loader:_load_cwl_from_stream:231 - Reading stream from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/develop/templates/stage-in-file.cwl...

2025-12-02 12:42:47.236 | DEBUG    | cwl_loader:load_cwl_from_stream:203 - CWL data of type <class 'ruamel.yaml.comments.CommentedMap'> successfully loaded from stream

2025-12-02 12:42:47.237 | DEBUG    | cwl_loader:load_cwl_from_yaml:130 - Updating the model from version 'v1.0' to version 'v1.2'...

2025-12-02 12:42:47.237 | DEBUG    | cwl_loader:load_cwl_from_yaml:141 - Raw CWL document successfully updated to v1.2!

2025-12-02 12:42:47.238 | DEBUG    | cwl_loader:load_cwl_from_yaml:145 - Parsing the raw CWL document to the CWL Utils DOM...

2025-12-02 12:42:47.404 | DEBUG    | cwl_loader:load_cwl_from_yaml:158 - Raw CWL document successfully parsed to the CWL Utils DOM!

2025-12-02 12:42:47.404 | DEBUG    | cwl_loader:load_cwl_from_yaml:160 - Dereferencing the steps[].run...

2025-12-02 12:42:47.405 | DEBUG    | cwl_loader:load_cwl_from_yaml:167 - steps[].run successfully dereferenced! Dereferencing the FQNs...

2025-12-02 12:42:47.405 | DEBUG    | cwl_loader:load_cwl_from_yaml:171 - CWL document successfully dereferenced! Now verifying steps[].run integrity...

2025-12-02 12:42:47.405 | DEBUG    | cwl_loader:load_cwl_from_yaml:175 - All steps[].run link are resolvable!

2025-12-02 12:42:47.406 | DEBUG    | cwl_loader:load_cwl_from_yaml:178 - Sorting Process instances by dependencies....

2025-12-02 12:42:47.406 | DEBUG    | cwl_loader:load_cwl_from_yaml:180 - Sorting process is over.

2025-12-02 12:42:47.407 | DEBUG    | cwl_loader:_load_cwl_from_stream:240 - Stream from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/develop/templates/stage-in-file.cwl successfully load!

2025-12-02 12:42:47.407 | DEBUG    | cwl_loader:load_cwl_from_location:228 - Loading CWL document from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/develop/templates/stage-out.cwl...

2025-12-02 12:42:47.440 | DEBUG    | cwl_loader:_load_cwl_from_stream:231 - Reading stream from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/develop/templates/stage-out.cwl...

2025-12-02 12:42:47.449 | DEBUG    | cwl_loader:load_cwl_from_stream:203 - CWL data of type <class 'ruamel.yaml.comments.CommentedMap'> successfully loaded from stream

2025-12-02 12:42:47.450 | DEBUG    | cwl_loader:load_cwl_from_yaml:130 - Updating the model from version 'v1.0' to version 'v1.2'...

2025-12-02 12:42:47.451 | DEBUG    | cwl_loader:load_cwl_from_yaml:141 - Raw CWL document successfully updated to v1.2!

2025-12-02 12:42:47.451 | DEBUG    | cwl_loader:load_cwl_from_yaml:145 - Parsing the raw CWL document to the CWL Utils DOM...

2025-12-02 12:42:47.633 | DEBUG    | cwl_loader:load_cwl_from_yaml:158 - Raw CWL document successfully parsed to the CWL Utils DOM!

2025-12-02 12:42:47.634 | DEBUG    | cwl_loader:load_cwl_from_yaml:160 - Dereferencing the steps[].run...

2025-12-02 12:42:47.634 | DEBUG    | cwl_loader:load_cwl_from_yaml:167 - steps[].run successfully dereferenced! Dereferencing the FQNs...

2025-12-02 12:42:47.635 | DEBUG    | cwl_loader:load_cwl_from_yaml:171 - CWL document successfully dereferenced! Now verifying steps[].run integrity...

2025-12-02 12:42:47.635 | DEBUG    | cwl_loader:load_cwl_from_yaml:175 - All steps[].run link are resolvable!

2025-12-02 12:42:47.636 | DEBUG    | cwl_loader:load_cwl_from_yaml:178 - Sorting Process instances by dependencies....

2025-12-02 12:42:47.636 | DEBUG    | cwl_loader:load_cwl_from_yaml:180 - Sorting process is over.

2025-12-02 12:42:47.636 | DEBUG    | cwl_loader:_load_cwl_from_stream:240 - Stream from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/develop/templates/stage-out.cwl successfully load!

2025-12-02 12:42:47.637 | INFO     | eoap_cwlwrap.types:_validate_stage_in:281 - Validating stage-in 'my-asthonishing-stage-in-directory'...

2025-12-02 12:42:47.639 | INFO     | eoap_cwlwrap.types:_validate_stage_in:303 - stage-in 'my-asthonishing-stage-in-directory' is valid

2025-12-02 12:42:47.639 | INFO     | eoap_cwlwrap.types:_validate_stage_in:281 - Validating stage-in 'my-asthonishing-stage-in-file'...

2025-12-02 12:42:47.640 | INFO     | eoap_cwlwrap.types:_validate_stage_in:303 - stage-in 'my-asthonishing-stage-in-file' is valid

2025-12-02 12:42:47.641 | INFO     | eoap_cwlwrap.types:validate_stage_out:339 - Validating stage-out 'my-super-stage-out'...

2025-12-02 12:42:47.641 | INFO     | eoap_cwlwrap.types:validate_stage_out:361 - stage-out 'my-super-stage-out' is valid

2025-12-02 12:42:47.642 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:101 - Building the CWL Orchestrator Workflow...

2025-12-02 12:42:47.642 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:130 - Analyzing pattern-10 inputs...

2025-12-02 12:42:47.643 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:146 - * pattern-10/aoi: string

2025-12-02 12:42:47.644 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:146 - * pattern-10/epsg: string

2025-12-02 12:42:47.644 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:146 - * pattern-10/indexes: string[]

2025-12-02 12:42:47.645 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:146 - * pattern-10/items: Directory[]

2025-12-02 12:42:47.645 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:159 -   Directory type detected, creating a related 'directory_stage_in_0'...

2025-12-02 12:42:47.646 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:161 -   Converting Directory[] to URL-compatible type...

2025-12-02 12:42:47.646 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:165 -   Directory[] converted to https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI[]

2025-12-02 12:42:47.647 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:186 -   Array detected, 'scatter' required for reference:items

2025-12-02 12:42:47.647 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:206 -   Connecting 'app/items' to 'directory_stage_in_0' output...

2025-12-02 12:42:47.648 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:257 - Analyzing pattern-10 outputs...

2025-12-02 12:42:47.649 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:263 - * pattern-10/vegetation_indexes: Directory[]

2025-12-02 12:42:47.649 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:268 -   Directory type detected, creating a related 'stage_out_0'...

2025-12-02 12:42:47.650 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:270 -   Converting Directory[] to URL-compatible type...

2025-12-02 12:42:47.653 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:274 -   Directory[] converted to https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI[]

2025-12-02 12:42:47.654 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:295 -   Array detected, scatter required for stac_catalog:app/vegetation_indexes

2025-12-02 12:42:47.654 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:315 -   Connecting 'app/vegetation_indexes' to 'stage_out_0' output...

2025-12-02 12:42:47.655 | INFO     | eoap_cwlwrap:_build_orchestrator_workflow:390 - Orchestrator Workflow built in 0.0134 seconds

No description has been provided for this image

Workflow components diagram¶

No description has been provided for this image

Inputs¶

Id	Type	Label	Doc
`aoi`	`string`	area of interest - pattern-10/aoi	area of interest as a bounding box - This parameter is derived from pattern-10/aoi
`epsg`	`string`	EPSG code - pattern-10/epsg	EPSG code - This parameter is derived from pattern-10/epsg
`indexes`	`string[]`	indexes - pattern-10/indexes	indexes to compute - This parameter is derived from pattern-10/indexes
`another_input`	`string`	Another Input - my-asthonishing-stage-in-directory/another_input	An additional input for demonstration purposes - This parameter is derived from my-asthonishing-stage-in-directory/another_input
`items`	`https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI[]`	Landsat-8/9 acquisition reference - pattern-10/items	Landsat-8/9 acquisition reference - This parameter is derived from pattern-10/items
`s3_bucket`	`string`	my-super-stage-out/s3_bucket	This parameter is derived from: my-super-stage-out/s3_bucket
`sub_path`	`string`	my-super-stage-out/sub_path	This parameter is derived from: my-super-stage-out/sub_path
`aws_access_key_id`	`string`	my-super-stage-out/aws_access_key_id	This parameter is derived from: my-super-stage-out/aws_access_key_id
`aws_secret_access_key`	`string`	my-super-stage-out/aws_secret_access_key	This parameter is derived from: my-super-stage-out/aws_secret_access_key
`region_name`	`string`	my-super-stage-out/region_name	This parameter is derived from: my-super-stage-out/region_name
`endpoint_url`	`string`	my-super-stage-out/endpoint_url	This parameter is derived from: my-super-stage-out/endpoint_url

Steps¶

Id	Runs	Label	Doc
`directory_stage_in_0`	`#my-asthonishing-stage-in-directory`	None	None
`app`	`#pattern-10`	None	None
`stage_out_0`	`#my-super-stage-out`	None	None

Outputs¶

Id	Type	Label	Doc
`vegetation_indexes`	`https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI[]`	Vegetation indexes	Vegetation indexes from Landsat-8/9 acquisitions

Execution¶

Inputs¶

{
  "aoi": "-118.985,38.432,-118.183,38.938",
  "epsg": "EPSG:4326",
  "indexes": [
    "ndvi",
    "ndwi"
  ],
  "items": [
    {
      "class": "https://raw.githubusercontent.com/eoap/schemas/main/url.yaml#URL",
      "value": "https://planetarycomputer.microsoft.com/api/stac/v1/collections/landsat-c2-l2/items/LC08_L2SP_042033_20231007_02_T1"
    },
    {
      "class": "https://raw.githubusercontent.com/eoap/schemas/main/url.yaml#URL",
      "value": "https://planetarycomputer.microsoft.com/api/stac/v1/collections/landsat-c2-l2/items/LC09_L2SP_042033_20231015_02_T1"
    }
  ]
}

Outputs¶

{
    "vegetation_indexes": [
        {
            "value": "s3://my-bucket/my/sub/path-3u3yv7/catalog.json",
            "type": "https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI"
        },
        {
            "value": "s3://my-bucket/my/sub/path-vs9ubo/catalog.json",
            "type": "https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI"
        },
        {
            "value": "s3://my-bucket/my/sub/path-6x30pl/catalog.json",
            "type": "https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI"
        },
        {
            "value": "s3://my-bucket/my/sub/path-8su0ae/catalog.json",
            "type": "https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI"
        }
    ]
}