... | ... | @@ -10,7 +10,7 @@ This is the CNMS's newest computing resource. |
|
|
We *own* ~**2400** cores. Many users must share these, think before you submit.
|
|
|
|
|
|
* Run test jobs to establish timing and parallelization parameters.
|
|
|
* Try to make your walltimes tight. (not always 48:00:00).
|
|
|
* Try to make your walltimes tight. (**not always 48:00:00**).
|
|
|
* Do not flood the queue with jobs, if you have many small jobs batch them. Ask how.
|
|
|
* We will actively thwart gaming the scheduling policies.
|
|
|
|
... | ... | @@ -63,11 +63,15 @@ Most plentiful node is Haswell 2x16, i.e. hw32 |
|
|
** MOAB Torque Cluster **
|
|
|
|
|
|
## Job Submission
|
|
|
There is only one non-experimental queue: batch
|
|
|
### queues
|
|
|
There are now two **queues** for cnms jobs.
|
|
|
* batch
|
|
|
* high_mem
|
|
|
|
|
|
You want to use the qos: **std**
|
|
|
|
|
|
Unless you have a short development job i.e. a test or debug run, then qos:**devel**
|
|
|
### Quality of service (QOS)
|
|
|
* std - generally this is what you want
|
|
|
* devel - short debug, build and experimental runs
|
|
|
* burst - premptable jobs that run on unused CONDO resources (you must request access from @epd)
|
|
|
|
|
|
If you need to run wide relatively short jobs, are experiencing long waits for std and can deal with them being occassionally prempted (i.e. killed) you can request access to qos: **burst** via [XCAMS](https://xcams.ornl.gov/xcams/groups/cades-cnms-burst)
|
|
|
|
... | ... | @@ -76,10 +80,8 @@ This is the obligatory PBS header for a job. |
|
|
``` shell
|
|
|
#!/bin/bash
|
|
|
#PBS -S /bin/bash
|
|
|
#PBS -m be
|
|
|
#PBS -M youremail@ornl.gov
|
|
|
#PBS -N job_name
|
|
|
#PBS -q batch
|
|
|
#PBS -N <YOUR_JOB_NAME>
|
|
|
#PBS -q <QUEUE_NAME>
|
|
|
#PBS -l nodes=2:ppn=32:hw32
|
|
|
#PBS -l walltime=00:30:00
|
|
|
#PBS -l naccesspolicy=singlejob
|
... | ... | @@ -127,15 +129,9 @@ Please read these, you can waste a great deal of resources if you do not underst |
|
|
|
|
|
## Advanced
|
|
|
|
|
|
### Quality of Service (QOS)
|
|
|
### Burst QOS
|
|
|
In theory there are two QOS levels useable on Cades:
|
|
|
|
|
|
* **std** -- allows use of up to our purchased amount of nodes.
|
|
|
|
|
|
* **burst** -- allows use of more nodes but the job can be preempted by std QOS jobs.
|
|
|
|
|
|
This is the basic header for a burst job
|
|
|
|
|
|
```shell
|
|
|
#!/bin/bash
|
|
|
#PBS -S /bin/bash
|
... | ... | |