Commit 7c70f33f authored by Maiterth, Matthias's avatar Maiterth, Matthias
Browse files

Merge branch 'net-dev-rebase' into 'develop' 

Rebase: Fix bluewaters dataloader working to demonstrate slowdown and refactor network

See merge request !110
parents 8fe97142 e1ad247a

README.md

+27 −0
Original line number Diff line number Diff line
@@ -169,6 +169,33 @@ See instructions in [server/README.md](https://code.ornl.gov/exadigit/simulation 


See instructions in [dashboard/README.md](https://code.ornl.gov/exadigit/simulation-dashboard)



## Running Tests



RAPS uses [pytest](https://docs.pytest.org/) for its test suite.  

Before running tests, ensure that you have a valid data directory available (e.g., `/opt/data`) and set the environment variable `RAPS_DATA_DIR` to point to it.



### Run all tests

```bash

RAPS_DATA_DIR=/opt/data pytest -n auto -x

```



By default, tests are parallelized with `pytest-xdist` (`-n auto`) to speed up execution.

The `-x` flag stops execution after the first failure. Add `-v` to run in verbose mode.



### Run only network-related tests



```bash

RAPS_DATA_DIR=/opt/data pytest -n auto -x -m network

```



See `pytest.ini` for the different options for `-m`.



### Run a specific test file



```bash

RAPS_DATA_DIR=/opt/data pytest tests/systems/test_engine.py

```



### Contributing Code



Install pre-commit hooks as set by the project:

config/bluewaters.yaml

+3 −1
Original line number Diff line number Diff line
@@ -49,7 +49,9 @@ scheduler: 
    NODE_FAIL: 0.01

network:

  topology: torus3d

  network_max_bw: 9600000000.0

  #topology: capacity

  #network_max_bw: 9.6E9

  network_max_bw: 1E7

  torus_x: 24

  torus_y: 24

  torus_z: 24

experiments/bluewaters.yaml

0 → 100644
+6 −0
Original line number Diff line number Diff line 
system: bluewaters

replay:

  - /opt/data/bluewaters

start: "20170328"

simulate_network: True

filter: "traffic > 1e8"

experiments/lassen.yaml

+1 −1
Original line number Diff line number Diff line 
system: lassen

replay:

  - ~/data/lassen/Lassen-Supercomputer-Job-Dataset

  - /opt/data/lassen/Lassen-Supercomputer-Job-Dataset

policy: fcfs

backfill: firstfit

fastforward: 365d

experiments/mit.yaml

+1 −1
Original line number Diff line number Diff line 
system: mit_supercloud

partitions: ["mit_supercloud/part-cpu", "mit_supercloud/part-gpu"]

replay:

  - ~/data/mit/202201

  - /opt/data/mit_supercloud

start: 2021-05-21T13:00

end: 2021-05-21T14:00