diff --git a/examples/atmospheric_correction-pansharpening-orthorectification/README.md b/examples/atmospheric_correction-pansharpening-orthorectification/README.md new file mode 100644 index 0000000000000000000000000000000000000000..f0b772efa81294f2bab18e14d2763dc7dac12312 --- /dev/null +++ b/examples/atmospheric_correction-pansharpening-orthorectification/README.md @@ -0,0 +1,103 @@ +# Configuration for: Atmospheric Correction, Pansharpening, and Orthorectification + +## Breaking Down the Configuration +- Specify the input directory or series of directories: + - Use the `input` value to parse through imagery in parallel. + - In the associated `config.json`, there is a single directory listed. Xylem will parse through this main directory and concurrently process all available subdirectories. + ```json + "input": + "/data/test-cases" + ``` + - Multiple directories may also be specified. Note that when listing multiple directories, they are listed within `[ ]`, signaling to Xylem these specific directories should be concurrently processed. + ```json + "input": [ + "/data/test-cases/test-case-GE01-medium-low-mid", + "/data/test-cases/test-case-WV02-medium-mid-high", + "/data/test-cases/test-case-wv03-large-high-mid", + "/data/test-cases/test-case-wv03-large-mid-low" + ] + ``` +- Define each module: + - Within the module variable, specify the `name` and `uri` to the module. The `uri` specifically informs Xylem from where to install the module environment. For example, if a user has Xylem installed and has cloned each of the modules in their `dev` directory within a Docker container, this path may look something like: + ```json + "uri": "file:///root/dev/module-a" + ``` +- Define each module's variables: + - Variables are unique to each module (detailed below). +- Simply follow the `template`: + - This section is built from the module's `Makefile`. Here, users can specify arguments and associated variables. Below is an example template for Module A: + ```json + "template": { + "command": "python", + "environment": { + "name": "module-a", + "manager": "conda" + }, + "arguments": [ + "-m", + "lib", + "--input_directory", + "{{ INPUT }}", + "--output_directory", + "{{ OUTPUT_DIRECTORY }}", + "--method", + "{{ METHOD }}", + "--profile", + "{{ PROFILE }}" + ] + } + ``` + +## Module Variables +For more thorough discussions on variables and overall structure, along with a link to technical documentation, please browse the README: +- [Module A](https://code-int.ornl.gov/gshs/common/imagery-processing/module-a/-/blob/main/README.md) +- [Module P](https://code-int.ornl.gov/gshs/common/imagery-processing/module-p/-/blob/main/README.md) +- [Module O](https://code-int.ornl.gov/gshs/common/imagery-processing/module-o/-/blob/main/README.md) + +### Module A: ***A***tmopsheric Correction +- `input_directory`: the directory containing the raw, Level 1B Maxar imagery. Module A currently expects a Maxar data directory structure. This argument will always be `INPUT` in the template. +- `output_directory`: the directory in which to store the output of Module A processing. +- `method`: specification for users to define if they want the output to contain top-of-atmosphere reflectance (`toa_reflectance`) or bottom-of-atmosphere reflectance (`boa_reflectance`). For the best representation of true surface reflectance (e.g., the removal of the blue effects of the atmosphere), users should select the latter. +- `profile`: specification for users to define the aersol profile selected in the Py6S model. Module A is currently optimized for `urban` applications, so consider using `maritime` sparingly. + - Future work for Module A will incorporate the maritime-optimized atmospheric correction efforts led by Matt McCarthy. + +### Module P: ***P***ansharpening +- `source_directory`: the directory containing the output of Module A. This argument should match the `output_directory` of Module A. +- `output_directory`: the directory in which to store the output of Module P processing. This argument should match both the `source_directory` for Module P and the `output_directory` of Module A. +- `method`: specification for users to define a pansharpening method. Currently, only `nn_diffuse` is supported. +- `module_list`: specification for users to identify all modules used in the given configuration. The presence, or absence, of the string `MODA` in this argument determines how the input directories are processed. In this example `config.json`, users would specify Modules A, P, and O. Options for this argument are: 'MODP', 'MODP, MODO', 'MODA, MODP', 'MODA, MODP, MODO'. + +### Module O: ***O***rthorectification +- 'source_directory': the directory containing the output of Module P. This argument should match the `output_directory` of Module P. +- 'output_directory': the directory in which to store the output of Module O processing. This argument should match both the `source_directory` for Module O and the `output_directory` of Module P. + +## Running the Module +- To run the standalone module independently of any workflow: + ```bash + make run + ``` +- To run the module with Xylem, at maximum verbosity, using the `config.json`: + ```bash + xylem run -vvvv + ``` + +## General Notes for a Multi-Module Configuration +- The general structure of the configuration remains the same: + - Specify the input directory or series of directories: + - There will still be the initial `input` value to include outside of the module definitions. This is generally going to be the directory (or specific subdirectories) of raw, level 1B imagery. + - Define the modules: + - Just as with a standalone configuration, users will need to identify all of the modules being used for processing, in the correct order and correct paths. + - Define each module's variables: + - All variables are defined as above. However, the only caveat with the multi-module implementation is aligning the input/source and output directories between modules. + - **The output of a given module will always be the same path as the input to the following module.** + - **All non-primary modules will have the same path for their source and output directories.** + - Follow the template: + - Just as with a standalone configuration, users can specify module arguments within each module's template. +- The multi-module configuration will provide outputs at each stage of processing, which are used as input to subsequent stages. This design is meant to support easier troubleshooting between modules, if needed. +- As imagery is processed, output files will be appended with the name of the processing module. Therefore, imagery processed with Modules A, P, and O will have filenames ending in `MODA_MODP_MODO`. + - Module P also updates the middle of the filenames from `P1BS` for panchromatic imagery and `M1BS` for multispectral imagery to `S3XS` which is a Maxar naming standard and a naming convention upheld throughout the use of Legacy PIPE. +- For multi-module processing, combinations can **only** be made in these two orders: + > Module A -> Module P -> Module O + + > Module P -> Module O + - Atmospheric correction is **always** the first processing step if it is included, and imagery **must** be pansharpened before it is orthorectified. \ No newline at end of file diff --git a/examples/atmospheric_correction-pansharpening-orthorectification/config.json b/examples/atmospheric_correction-pansharpening-orthorectification/config.json new file mode 100644 index 0000000000000000000000000000000000000000..2f69578a2c2be2b65fb4b99a0eb0e0f84b030c3d --- /dev/null +++ b/examples/atmospheric_correction-pansharpening-orthorectification/config.json @@ -0,0 +1,99 @@ +{ + "version": "0.0.9", + "description": "Test workflow configuration", + "requirements": [ + "python", + "conda" + ], + "keywords": [], + "input": + "/data/test-cases", + "modules": [ + { + "name": "Module A (local)", + "type": "script", + "programmingLanguage": "python", + "uri": "file:///path/to/module-a", + "variables": { + "OUTPUT_DIRECTORY": "/data/output", + "METHOD": "boa_reflectance", + "PROFILE": "urban" + }, + "template": { + "command": "python", + "environment": { + "name": "module-a", + "manager": "conda" + }, + "arguments": [ + "-m", + "lib", + "--input_directory", + "{{ INPUT }}", + "--output_directory", + "{{ OUTPUT_DIRECTORY }}", + "--method", + "{{ METHOD }}", + "--profile", + "{{ PROFILE }}" + ] + } + }, + { + "name": "Module P (local)", + "type": "script", + "programmingLanguage": "python", + "uri": "file:///path/to/module-p", + "variables": { + "SOURCE_DIRECTORY": "/data/output", + "OUTPUT_DIRECTORY": "/data/output", + "METHOD": "nn_diffuse", + "MODULE_LIST": "MODA, MODP, MODO" + }, + "template": { + "command": "python", + "environment": { + "name": "module-p", + "manager": "conda" + }, + "arguments": [ + "-m", + "lib", + "--source_directory", + "{{ SOURCE_DIRECTORY }}", + "--output_directory", + "{{ OUTPUT_DIRECTORY }}", + "--method", + "{{ METHOD }}", + "--module_list", + "{{ MODULE_LIST }}" + ] + } + }, + { + "name": "Module O (local)", + "type": "script", + "programmingLanguage": "python", + "uri": "file:///path/to/module-o", + "variables": { + "SOURCE_DIRECTORY": "/data/output", + "OUTPUT_DIRECTORY": "/data/output" + }, + "template": { + "command": "python", + "environment": { + "name": "module-o", + "manager": "conda" + }, + "arguments": [ + "-m", + "lib", + "--source_directory", + "{{ SOURCE_DIRECTORY }}", + "--output_directory", + "{{ OUTPUT_DIRECTORY }}" + ] + } + } + ] +} \ No newline at end of file diff --git a/examples/atmospheric_correction/README.md b/examples/atmospheric_correction/README.md new file mode 100644 index 0000000000000000000000000000000000000000..04f5b4dd14450f1f6a5f9ae892c74b9521a7719c --- /dev/null +++ b/examples/atmospheric_correction/README.md @@ -0,0 +1,70 @@ +# Configuration for: Atmospheric Correction + +## Breaking Down the Configuration +- Specify the input directory or series of directories: + - Use the `input` value to parse through imagery in parallel. + - In the associated `config.json`, there is a single directory listed. Xylem will parse through this main directory and concurrently process all available subdirectories. + ```json + "input": + "/data/test-cases" + ``` + - Multiple directories may also be specified. Note that when listing multiple directories, they are listed within `[ ]`, signaling to Xylem these specific directories should be concurrently processed. + ```json + "input": [ + "/data/test-cases/test-case-GE01-medium-low-mid", + "/data/test-cases/test-case-WV02-medium-mid-high", + "/data/test-cases/test-case-wv03-large-high-mid", + "/data/test-cases/test-case-wv03-large-mid-low" + ] + ``` +- Define the module: + - Within the module variable, specify the `name` and `uri` to the module. The `uri` specifically informs Xylem from where to install the module environment. For example, if a user has Xylem installed and has cloned each of the modules in their `dev` directory within a Docker container, this path may look something like: + ```json + "uri": "file:///root/dev/module-a" + ``` +- Define the module variables: + - Variables are unique to each module (detailed below). +- Simply follow the `template`: + - This section is built from the module's `Makefile`. Here, users can specify arguments and associated variables. Below is an example template for Module A: + ```json + "template": { + "command": "python", + "environment": { + "name": "module-a", + "manager": "conda" + }, + "arguments": [ + "-m", + "lib", + "--input_directory", + "{{ INPUT }}", + "--output_directory", + "{{ OUTPUT_DIRECTORY }}", + "--method", + "{{ METHOD }}", + "--profile", + "{{ PROFILE }}" + ] + } + ``` + +## Module Variables +For more thorough discussions on variables and overall structure, along with a link to technical documentation, please browse the README: +- [Module A](https://code-int.ornl.gov/gshs/common/imagery-processing/module-a/-/blob/main/README.md) + +### Module A: ***A***tmopsheric Correction +- `input_directory`: the directory containing the raw, Level 1B Maxar imagery. Module A currently expects a Maxar data directory structure. Because Module A is either run as a standalone module or always the first in a multi-module sequence, this argument will always be `INPUT` in the template. +- `output_directory`: the directory in which to store the output of Module A processing. +- `method`: specification for users to define if they want the output to contain top-of-atmosphere reflectance (`toa_reflectance`) or bottom-of-atmosphere reflectance (`boa_reflectance`). For the best representation of true surface reflectance (e.g., the removal of the blue effects of the atmosphere), users should select the latter. +- `profile`: specification for users to define the aersol profile selected in the Py6S model. Module A is currently optimized for `urban` applications, so consider using `maritime` sparingly. + - Future work for Module A will incorporate the maritime-optimized atmospheric correction efforts led by Matt McCarthy. + +## Running the Module +- To run the standalone module independently of any workflow: + ```bash + make run + ``` +- To run the module with Xylem, at maximum verbosity, using the `config.json`: + ```bash + xylem run -vvvv + ``` \ No newline at end of file diff --git a/examples/atmospheric_correction/config.json b/examples/atmospheric_correction/config.json new file mode 100644 index 0000000000000000000000000000000000000000..9d2db551f1fd97be81f83d97f55866fac95df5d8 --- /dev/null +++ b/examples/atmospheric_correction/config.json @@ -0,0 +1,43 @@ +{ + "version": "0.0.9", + "description": "Test workflow configuration", + "requirements": [ + "python", + "conda" + ], + "keywords": [], + "input": + "/data/test-cases", + "modules": [ + { + "name": "Module A (local)", + "type": "script", + "programmingLanguage": "python", + "uri": "file:///path/to/module-a", + "variables": { + "OUTPUT_DIRECTORY": "/data/output", + "METHOD": "boa_reflectance", + "PROFILE": "urban" + }, + "template": { + "command": "python", + "environment": { + "name": "module-a", + "manager": "conda" + }, + "arguments": [ + "-m", + "lib", + "--input_directory", + "{{ INPUT }}", + "--output_directory", + "{{ OUTPUT_DIRECTORY }}", + "--method", + "{{ METHOD }}", + "--profile", + "{{ PROFILE }}" + ] + } + } + ] +} \ No newline at end of file diff --git a/examples/orthorectification/README.md b/examples/orthorectification/README.md new file mode 100644 index 0000000000000000000000000000000000000000..cad6cb91127cbc47f8f10c46fe38bf755d946d1c --- /dev/null +++ b/examples/orthorectification/README.md @@ -0,0 +1,63 @@ +# Configuration for: Orthorectification + +## Breaking Down the Configuration +- Specify the input directory or series of directories: + - Use the `input` value to parse through imagery in parallel. + - In the associated `config.json`, multiple directories are listed. Note that when listing multiple directories, they are listed within `[ ]`, signaling to Xylem these specific directories should be concurrently processed. + ```json + "input": [ + "/data/test-cases/test-case-GE01-medium-low-mid", + "/data/test-cases/test-case-WV02-medium-mid-high", + "/data/test-cases/test-case-wv03-large-high-mid", + "/data/test-cases/test-case-wv03-large-mid-low" + ] + ``` + - A single directory may also be specified. Xylem will parse through this main directory and concurrently process all available subdirectories. + ```json + "input": + "/data/test-cases" + ``` +- Define the module: + - Within the module variable, specify the `name` and `uri` to the module. The `uri` specifically informs Xylem from where to install the module environment. For example, if a user has Xylem installed and has cloned each of the modules in their `dev` directory within a Docker container, this path may look something like: + ```json + "uri": "file:///root/dev/module-o" + ``` +- Define the module variables: + - Variables are unique to each module (detailed below). +- Simply follow the `template`: + - This section is built from the module's `Makefile`. Here, users can specify arguments and associated variables. Below is an example template for Module O: + ```json + "template": { + "command": "python", + "environment": { + "name": "module-o", + "manager": "conda" + }, + "arguments": [ + "-m", + "lib", + "--source_directory", + "{{ INPUT }}", + "--output_directory", + "{{ OUTPUT_DIRECTORY }}" + ] + } + ``` + +## Module Variables +For more thorough discussions on variables and overall structure, along with a link to technical documentation, please browse the README: +- [Module O](https://code-int.ornl.gov/gshs/common/imagery-processing/module-o/-/blob/main/README.md) + +### Module O: ***O***rthorectification +- 'source_directory': the directory containing the output of Module P. Module O can be run as a unique module following the example `config.json`. HOWEVER, the input MUST have already been processed through Module P. +- 'output_directory': the directory in which to store the output of Module O processing. This argument should match the `source_directory` for Module O. + +## Running the Module +- To run the standalone module independently of any workflow: + ```bash + make run + ``` +- To run the module with Xylem, at maximum verbosity, using the `config.json`: + ```bash + xylem run -vvvv + ``` \ No newline at end of file diff --git a/examples/orthorectification/config.json b/examples/orthorectification/config.json new file mode 100644 index 0000000000000000000000000000000000000000..88f49f52f39a23477980cd3feb24f0490721b534 --- /dev/null +++ b/examples/orthorectification/config.json @@ -0,0 +1,40 @@ +{ + "version": "0.0.9", + "description": "Test workflow configuration", + "requirements": [ + "python", + "conda" + ], + "input": [ + "/data/test-cases/test-case-GE01-medium-low-mid", + "/data/test-cases/test-case-WV02-medium-mid-high", + "/data/test-cases/test-case-wv03-large-high-mid", + "/data/test-cases/test-case-wv03-large-mid-low" + ], + "modules": [ + { + "name": "Module O (local)", + "type": "script", + "programmingLanguage": "python", + "uri": "file:///path/to/module-o", + "variables": { + "OUTPUT_DIRECTORY": "/data/output" + }, + "template": { + "command": "python", + "environment": { + "name": "module-o", + "manager": "conda" + }, + "arguments": [ + "-m", + "lib", + "--source_directory", + "{{ INPUT }}", + "--output_directory", + "{{ OUTPUT_DIRECTORY }}" + ] + } + } + ] +} \ No newline at end of file diff --git a/examples/pansharpening-orthorectification/README.md b/examples/pansharpening-orthorectification/README.md new file mode 100644 index 0000000000000000000000000000000000000000..b2e6982708c6a1ff9aea0fcfde81327d9f617198 --- /dev/null +++ b/examples/pansharpening-orthorectification/README.md @@ -0,0 +1,95 @@ +# Configuration for: Pansharpening and Orthorectification + +## Breaking Down the Configuration +- Specify the input directory or series of directories: + - Use the `input` value to parse through imagery in parallel. + - In the associated `config.json`, there is a single directory listed. Xylem will parse through this main directory and concurrently process all available subdirectories. + ```json + "input": + "/data/test-cases" + ``` + - Multiple directories may also be specified. Note that when listing multiple directories, they are listed within `[ ]`, signaling to Xylem these specific directories should be concurrently processed. + ```json + "input": [ + "/data/test-cases/test-case-GE01-medium-low-mid", + "/data/test-cases/test-case-WV02-medium-mid-high", + "/data/test-cases/test-case-wv03-large-high-mid", + "/data/test-cases/test-case-wv03-large-mid-low" + ] + ``` +- Define each module: + - Within the module variable, specify the `name` and `uri` to the module. The `uri` specifically informs Xylem from where to install the module environment. For example, if a user has Xylem installed and has cloned each of the modules in their `dev` directory within a Docker container, this path may look something like: + ```json + "uri": "file:///root/dev/module-p" + ``` +- Define each module's variables: + - Variables are unique to each module (detailed below). +- Simply follow the `template`: + - This section is built from the module's `Makefile`. Here, users can specify arguments and associated variables. Below is an example template for Module P: + ```json + "template": { + "command": "python", + "environment": { + "name": "module-p", + "manager": "conda" + }, + "arguments": [ + "-m", + "lib", + "--source_directory", + "{{ INPUT }}", + "--output_directory", + "{{ OUTPUT_DIRECTORY }}", + "--method", + "{{ METHOD }}", + "--module_list", + "{{ MODULE_LIST }}" + ] + } + ``` + +## Module Variables +For more thorough discussions on variables and overall structure, along with a link to technical documentation, please browse the README: +- [Module P](https://code-int.ornl.gov/gshs/common/imagery-processing/module-p/-/blob/main/README.md) +- [Module O](https://code-int.ornl.gov/gshs/common/imagery-processing/module-o/-/blob/main/README.md) + +### Module P: ***P***ansharpening +- `source_directory`: the directory containing the raw, Level 1B Maxar imagery (or the output of Module A, if the images have been previously processed through Module A). +- `output_directory`: the directory in which to store the output of Module P processing. +- `method`: specification for users to define a pansharpening method. Currently, only `nn_diffuse` is supported. +- `module_list`: specification for users to identify all modules used in the given configuration. The presence, or absence, of the string `MODA` in this argument determines how the input directories are processed. In this example `config.json`, users would specify Modules P and O, assuming the imagery was not previously processing with Module A. Options for this argument are: 'MODP', 'MODP, MODO', 'MODA, MODP', 'MODA, MODP, MODO'. + +### Module O: ***O***rthorectification +- 'source_directory': the directory containing the output of Module P. This argument should match the `output_directory` of Module P. +- 'output_directory': the directory in which to store the output of Module O processing. This argument should match both the `source_directory` for Module O and the `output_directory` of Module P. + +## Running the Module +- To run the standalone module independently of any workflow: + ```bash + make run + ``` +- To run the module with Xylem, at maximum verbosity, using the `config.json`: + ```bash + xylem run -vvvv + ``` + +## General Notes for a Multi-Module Configuration +- The general structure of the configuration remains the same: + - Specify the input directory or series of directories: + - There will still be the initial `input` value to include outside of the module definitions. This is generally going to be the directory (or specific subdirectories) of raw, level 1B imagery. + - Define the modules: + - Just as with a standalone configuration, users will need to identify all of the modules being used for processing, in the correct order and correct paths. + - Define each module's variables: + - All variables are defined as above. However, the only caveat with the multi-module implementation is aligning the input/source and output directories between modules. + - **The output of a given module will always be the same path as the input to the following module.** + - **All non-primary modules will have the same path for their source and output directories.** + - Follow the template: + - Just as with a standalone configuration, users can specify module arguments within each module's template. +- The multi-module configuration will provide outputs at each stage of processing, which are used as input to subsequent stages. This design is meant to support easier troubleshooting between modules, if needed. +- As imagery is processed, output files will be appended with the name of the processing module. Therefore, imagery processed with Modules P and O will have filenames ending in `MODP_MODO`. + - Module P also updates the middle of the filenames from `P1BS` for panchromatic imagery and `M1BS` for multispectral imagery to `S3XS` which is a Maxar naming standard and a naming convention upheld throughout the use of Legacy PIPE. +- For multi-module processing, combinations can **only** be made in these two orders: + > Module A -> Module P -> Module O + + > Module P -> Module O + - Atmospheric correction is **always** the first processing step if it is included, and imagery **must** be pansharpened before it is orthorectified. \ No newline at end of file diff --git a/examples/pansharpening-orthorectification/config.json b/examples/pansharpening-orthorectification/config.json new file mode 100644 index 0000000000000000000000000000000000000000..2ee5c58fef4def914e477a98beb5d4a055a794f8 --- /dev/null +++ b/examples/pansharpening-orthorectification/config.json @@ -0,0 +1,68 @@ +{ + "version": "0.0.9", + "description": "Test workflow configuration", + "requirements": [ + "python", + "conda" + ], + "keywords": [], + "input": + "/data/test-cases", + "modules": [ + { + "name": "Module P (local)", + "type": "script", + "programmingLanguage": "python", + "uri": "file:///path/to/module-p", + "variables": { + "OUTPUT_DIRECTORY": "/data/output", + "METHOD": "nn_diffuse", + "MODULE_LIST": "MODP, MODO" + }, + "template": { + "command": "python", + "environment": { + "name": "module-p", + "manager": "conda" + }, + "arguments": [ + "-m", + "lib", + "--source_directory", + "{{ INPUT }}", + "--output_directory", + "{{ OUTPUT_DIRECTORY }}", + "--method", + "{{ METHOD }}", + "--module_list", + "{{ MODULE_LIST }}" + ] + } + }, + { + "name": "Module O (local)", + "type": "script", + "programmingLanguage": "python", + "uri": "file:///path/to/module-o", + "variables": { + "SOURCE_DIRECTORY": "/data/output", + "OUTPUT_DIRECTORY": "/data/output" + }, + "template": { + "command": "python", + "environment": { + "name": "module-o", + "manager": "conda" + }, + "arguments": [ + "-m", + "lib", + "--source_directory", + "{{ SOURCE_DIRECTORY }}", + "--output_directory", + "{{ OUTPUT_DIRECTORY }}" + ] + } + } + ] +} \ No newline at end of file diff --git a/examples/pansharpening/README.md b/examples/pansharpening/README.md new file mode 100644 index 0000000000000000000000000000000000000000..7301a6505202a48561ba1db345a806eaff63a019 --- /dev/null +++ b/examples/pansharpening/README.md @@ -0,0 +1,69 @@ +# Configuration for: Pansharpening + +## Breaking Down the Configuration +- Specify the input directory or series of directories: + - Use the `input` value to parse through imagery in parallel. + - In the associated `config.json`, multiple directories are listed. Note that when listing multiple directories, they are listed within `[ ]`, signaling to Xylem these specific directories should be concurrently processed. + ```json + "input": [ + "/data/test-cases/test-case-GE01-medium-low-mid", + "/data/test-cases/test-case-WV02-medium-mid-high", + "/data/test-cases/test-case-wv03-large-high-mid", + "/data/test-cases/test-case-wv03-large-mid-low" + ] + ``` + - A single directory may also be specified. Xylem will parse through this main directory and concurrently process all available subdirectories. + ```json + "input": + "/data/test-cases" + ``` +- Define the module: + - Within the module variable, specify the `name` and `uri` to the module. The `uri` specifically informs Xylem from where to install the module environment. For example, if a user has Xylem installed and has cloned each of the modules in their `dev` directory within a Docker container, this path may look something like: + ```json + "uri": "file:///root/dev/module-p" + ``` +- Define the module variables: + - Variables are unique to each module (detailed below). +- Simply follow the `template`: + - This section is built from the module's `Makefile`. Here, users can specify arguments and associated variables. Below is an example template for Module P: + ```json + "template": { + "command": "python", + "environment": { + "name": "module-p", + "manager": "conda" + }, + "arguments": [ + "-m", + "lib", + "--source_directory", + "{{ INPUT }}", + "--output_directory", + "{{ OUTPUT_DIRECTORY }}", + "--method", + "{{ METHOD }}", + "--module_list", + "{{ MODULE_LIST }}" + ] + } + ``` + +## Module Variables +For more thorough discussions on variables and overall structure, along with a link to technical documentation, please browse the README: +- [Module P](https://code-int.ornl.gov/gshs/common/imagery-processing/module-p/-/blob/main/README.md) + +### Module P: ***P***ansharpening +- `source_directory`: the directory containing the raw, Level 1B Maxar imagery (or the output of Module A, if the images have been previously processed through Module A). When Module P is run as a standalone module, as with this `config.json`, this argument will be `INPUT` in the template. +- `output_directory`: the directory in which to store the output of Module P processing. +- `method`: specification for users to define a pansharpening method. Currently, only `nn_diffuse` is supported. +- `module_list`: specification for users to identify all modules used in the given configuration. The presence, or absence, of the string `MODA` in this argument determines how the input directories are processed. If the imagery has already been processed with Module A, include Module A on the list, even if you are only processing with Module P following the example `config.json`. Options for this argument are: 'MODP', 'MODP, MODO', 'MODA, MODP', 'MODA, MODP, MODO'. + +## Running the Module +- To run the standalone module independently of any workflow: + ```bash + make run + ``` +- To run the module with Xylem, at maximum verbosity, using the `config.json`: + ```bash + xylem run -vvvv + ``` \ No newline at end of file diff --git a/examples/pansharpening/config.json b/examples/pansharpening/config.json new file mode 100644 index 0000000000000000000000000000000000000000..e093ebc14e204fc4e76433419ba45ce2090ccabe --- /dev/null +++ b/examples/pansharpening/config.json @@ -0,0 +1,46 @@ +{ + "version": "0.0.9", + "description": "Test workflow configuration", + "requirements": [ + "python", + "conda" + ], + "input": [ + "/data/test-cases/test-case-GE01-medium-low-mid", + "/data/test-cases/test-case-WV02-medium-mid-high", + "/data/test-cases/test-case-wv03-large-high-mid", + "/data/test-cases/test-case-wv03-large-mid-low" + ], + "modules": [ + { + "name": "Module P (local)", + "type": "script", + "programmingLanguage": "python", + "uri": "file:///path/to/module-p", + "variables": { + "OUTPUT_DIRECTORY": "/data/output", + "METHOD": "nn_diffuse", + "MODULE_LIST": "MODP" + }, + "template": { + "command": "python", + "environment": { + "name": "module-p", + "manager": "conda" + }, + "arguments": [ + "-m", + "lib", + "--source_directory", + "{{ INPUT }}", + "--output_directory", + "{{ OUTPUT_DIRECTORY }}", + "--method", + "{{ METHOD }}", + "--module_list", + "{{ MODULE_LIST }}" + ] + } + } + ] +} \ No newline at end of file