Skip to content
GitLab
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Sign in
Toggle navigation
Menu
Open sidebar
Tsaris, Aristeidis (aris)
pytorch_tutorial
Commits
4483a884
Commit
4483a884
authored
Oct 25, 2021
by
Aristeidis Tsaris
Browse files
adding different env scripts
parent
d440969a
Changes
2
Hide whitespace changes
Inline
Side-by-side
ascent/sub_ibmwml.lsf
0 → 100755
View file @
4483a884
#!/bin/bash
# Begin LSF directives
#BSUB -P stf011
#BSUB -J sc21
#BSUB -o logs/sc21.o%J
#BSUB -W 0:30
#BSUB -nnodes 1
#BSUB -alloc_flags "nvme smt4"
####BSUB -N
# End LSF directives and begin shell commands
nnodes
=
$(
cat
${
LSB_DJOB_HOSTFILE
}
|
sort
|
uniq
|
grep
-v
login |
grep
-v
batch |
wc
-l
)
DATA_DIR
=
/gpfs/wolf/gen166/proj-shared/atsaris/imagenet/data/
LOG_DIR
=
logs/
#source /gpfs/wolf/stf011/proj-shared/28t/env/open-ce/activate.sh
source
/gpfs/wolf/stf011/proj-shared/28t/env/ibm-wml/activate.sh
jsrun
--smpiargs
=
"-disable_gpu_hooks"
-n
${
nnodes
}
-a1
-c42
-g1
-r1
\
--bind
=
proportional-packed:7
--launch_distribution
=
packed
\
bash
-c
"
\
source export_DDP_envvars.sh &&
\
python -u ../imagenet/main.py
\
--arch resnet50
\
-j 28
\
-p 10
\
-b 128
\
--training-only
\
--raport-file
${
LOG_DIR
}
/synthetic.ibmwml.1GPU.json
\
--epochs 1
\
--prof 100
\
--no-checkpoints
\
--data-backend sythetic
\
--amp
\
--memory-format nhwc
\
${
DATA_DIR
}
"
ascent/sub_opence.lsf
0 → 100755
View file @
4483a884
#!/bin/bash
# Begin LSF directives
#BSUB -P stf011
#BSUB -J sc21
#BSUB -o logs/sc21.o%J
#BSUB -W 0:30
#BSUB -nnodes 1
#BSUB -alloc_flags "nvme smt4"
####BSUB -N
# End LSF directives and begin shell commands
nnodes
=
$(
cat
${
LSB_DJOB_HOSTFILE
}
|
sort
|
uniq
|
grep
-v
login |
grep
-v
batch |
wc
-l
)
DATA_DIR
=
/gpfs/wolf/gen166/proj-shared/atsaris/imagenet/data/
LOG_DIR
=
logs/
source
/gpfs/wolf/stf011/proj-shared/28t/env/open-ce/activate.sh
#source /gpfs/wolf/stf011/proj-shared/28t/env/ibm-wml/activate.sh
jsrun
--smpiargs
=
"-disable_gpu_hooks"
-n
${
nnodes
}
-a1
-c42
-g1
-r1
\
--bind
=
proportional-packed:7
--launch_distribution
=
packed
\
bash
-c
"
\
source export_DDP_envvars.sh &&
\
python -u ../imagenet/main.py
\
--arch resnet50
\
-j 28
\
-p 10
\
-b 128
\
--training-only
\
--raport-file
${
LOG_DIR
}
/pytorch.opence.1GPU.json
\
--epochs 1
\
--prof 100
\
--no-checkpoints
\
--data-backend pytorch
\
--amp
\
--memory-format nhwc
\
${
DATA_DIR
}
"
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment