Commit 9082a113 authored by Wohlgemuth, Jason's avatar Wohlgemuth, Jason
Browse files

feat: Add architecture document

parent 4da18fa0

ARCHITECTURE.md

0 → 100644
+34 −0
Original line number Diff line number Diff line 
# Architecture

> This file describes the high-level architecture and some specific design decisions of PIPE



## Components

- `pipe-cli` (CLI Application)

  - Depends on `pipe-lib`

  - Depends on [clap](https://docs.rs/clap/latest/clap/) crate for CLI arguments

  - Contains code for CLI [commands](./pipe-cli/src/commands/) (e.g., `run`, `check`, etc...) and [reporters](./pipe-cli/src/reporters/)

- `pipe-lib` (Rust code)

  - Utility functions

  - Code for handling user input, ingesting configuration data, running workflows, etc...

- `pipe-py` (Python library)

  - 🚧 Under construction

  - Export Rust code as Python library using [PyO3/Maturin](https://github.com/PyO3/maturin)

  - Export select `pipe-lib` functions as Python functions



## Diagram

```mermaid

graph LR

    a[PIPE] --> b[pipe-cli/]

    a --> c[pipe-lib/]

    a --> d[pipe-py/]

    b --> e[CLI application]

    c --> f[Rust crate]

    d --> g[Python</br>package]

    e --> h[commands]

    e --> i[reporters]

    f --> j[pipe-cli</br>utility functions]

    g --> k[Module</br>utility functions]

```



## Notes

- Tests are written for modules in adjacent `tests.rs` files within the associated module directory

- Generated documentation is published to [https://pipe.ornl.gov](https://pipe.ornl.gov)
 
 No newline at end of file

CONTRIBUTING.md

+0 −15
Original line number Diff line number Diff line
@@ -43,21 +43,6 @@ 
    make test-cross

    ```



## Project Structure

```mermaid

graph LR

    a[PIPE] --> b[pipe-cli/]

    a --> c[pipe-lib/]

    a --> d[pipe-py/]

    b --> e[CLI application]

    c --> f[Rust crate]

    d --> g[Python</br>package]

    e --> h[commands]

    e --> i[reporters]

    f --> j[pipe-cli</br>utility functions]

    g --> k[Module</br>utility functions]

```



## Coding Guidelines

- Logging

    - This project uses the [tracing](https://docs.rs/tracing/latest/tracing/) crate for logging

README.md

+3 −0
Original line number Diff line number Diff line
@@ -16,6 +16,9 @@ Parallel Integration & Processing Engine 
## Documentation

> Read the code docs at https://pipe.code.ornl and/or explore the [Wiki](https://code.ornl.gov/groups/GSHS/common/pipe/-/wikis/home)



### Architecture

> See [ARCHITECTURE.md](./ARCHITECTURE.md)



## Roadmap

> See [ROADMAP.md](./ROADMAP.md)

pipe-cli/src/commands/run/mod.rs

+8 −4
Original line number Diff line number Diff line
@@ -23,7 +23,7 @@ pub fn run( 
    _output: &Option<PathBuf>,

    dry_run: bool,

    offline: bool,

    _skip_verify_checksum: bool,

    skip_verify_checksum: bool,

    skip_verify_requirements: bool,

    verbose: Verbosity,

) -> Result<(), Report> {
@@ -60,7 +60,7 @@ pub fn run( 
                    }

                });

            } else {

                debug!("Skipping requirements verification");

                warn!("Skipping requirements verification");

            }

            // Prepare modules

            cfg.modules.par_iter().for_each(|module| match &module.module_type {
@@ -72,8 +72,12 @@ pub fn run( 
                            // TODO: Filter URL using whitelist

                            // TODO: Check that URL exists

                            // TODO: Download binary to temp directory

                            if !skip_verify_checksum {

                                // TODO: Verify binary checksum

                                // TODO: Verify binary is executable (https://docs.rs/is_executable/latest/is_executable/)

                            } else {

                                warn!("Skipping checksum verification");

                            }

                            unimplemented!("Remote binary modules are not supported yet");

                        }

                        | _ => unimplemented!("Only HTTP, HTTPS, and SSH are currently supported for remote binary modules"),