Rebase: Fix bluewaters dataloader working to demonstrate slowdown and refactor network

Improved a number of issues with getting bluewaters dataloader working correctly with bluewaters dataset, and refactor network.py to now be a network directory with separate modules representing different types of network topologies: dragonfly, fat_tree, and torus3d. Had to add capacity network topology option into raps/config.py. Add --filter option for filtering a subset of jobs to load, e.g., --filter "traffic > 1e8" will only select jobs which have network traffic > 1e8 bytes per node. We'll probably need to add some safeguards for this in the future and move it into its own module,e .g.:

filter_str = "traffic > 1e8 and nodes_required > 10"

# Safely evaluate only in the context of known variables
allowed_locals = {
    "traffic": traffic,
    "nodes": nodes,
    "wall_time": wall_time
}
keep_jobs = eval(filter_str, {"__builtins__": {}}, allowed_locals)

Rebased onto develop

Merge request reports

Loading