# Setting up Spack for Plato on Summit Spack generally requires special configuration to work correctly on Summit, primarily because Summit's architecture is uncommon among the authors of spack packages. The necessary spack configs are in `/etc/spack` of this repo. A fork of the OpenBLAS package that has been extended to work better on PPC64LE architecture is also included in `/repos/ppc64le` which is referenced by the spack `repos.yaml` configuration provided using relative paths. The script `setup-spack-for-plato.sh` walks through the process of setting up a spack instance and is annotated to provide a little explanation for each step. Each of the spack configs are also lightly annotated to explain why some settings should be what they are. It should be entirely possible to, on Summit, call: ``` git clone https://code.ornl.gov/m9b/plato-on-summit.git plato cd plato ./setup-spack-for-plato.sh ``` to build Plato on Summit using gcc v4.8.5 without the CUDA variant. Using the resulting spack instance in the future to build other variants of Plato interactively can be done with: ``` cd plato # directory where repo was cloned module purge # mv $HOME/.spack $HOME/.spack.backup # optional if having problems . ./spack/share/spack/setup-env.sh spack install ... ``` ## Regarding OpenBLAS in Specs The comments in `setup-spack-for-plato.sh` regarding OpenBLAS are important. The default compiler used when no compiler is given as part of the spec is set in `packages.yaml` to be the OS-provided GCC v4.8.5 toolchain in `/usr/bin`. This is done because packages built with the OS toolchain don't require compiler modules to be loaded due to libstdc++ issues. Spack builds use RPATHs and are generally less affected by dependency modules, but spack builds are not immune from occasionally needing compiler-specific environment settings provided through environment modules at runtime. However, OpenBLAS 0.3.6 introduced Power9 optimizations which use flags unavailable to GCC4. Therefore, it is necessary to build OpenBLAS v0.3.5 with `cpu_target=POWER8` when using GCC v5.X or older. The default version/cpu_target option can be set in `packages.yaml` if the user knows which toolchain (GCC v6 or newer or GCC v5.x or older) they will generally be using. The default OpenBLAS options set by the provided `packages.yaml` config file assumes the use of GCC6 or newer when not explicitly set via a package spec. Because the default toolchain is GCC 4.8.5 and the default options for OpenBLAS assume the use of GCC6 or newer, it is generally necessary to declare the OpenBLAS version and cpu_target OR compiler (if GCC6 or newer) in the package spec if it depends on BLAS. Examples: No compiler spec, so OpenBLAS version and target must be specified because GCC4 will be used. ``` spack spec -lINt platoengine ^ppc64le.openblas@0.3.5 cpu_target=POWER8 ``` GCC7 being used, so OpenBLAS defaults in `packages.yaml` take effect and are valid. ``` spack spec -lINt platoengine%gcc@7.4.0 ```