Skip to content

Jenkins: Performance data upload from Compy, Cori

Mametjanov, Azamat requested to merge sarats/pace-compy into master

Created by: sarats

This script facilitates automatic data upload from Compy's performance archive into PACE.

It requires the Jenkins user to run this manually the first time to use the Github credentials to create an authorized token for later uploads automatically.

It stores the token information in $HOME/.pacecc.

It does not require cloning of E3SM sources or builds.

It moves certain files from the machine's PERF_ARCHIVE_DIR to the older (OLD_PERF_ARCHIVE_DIR) after data is processed.

It deletes very large files (>50MB) that are not meant to be placed in the performance archive by design.

Testing: Successfully uploaded performance data from Compy by commenting out the boilerplate initialization. #source $SCRIPTROOT/util/setup_common.sh I didn't load any Python module, system Python was sufficient. In case of Python environment issues, just unload any custom Python envs.

Merge request reports