Commit 4571e507 authored by Brown, Jesse's avatar Brown, Jesse Committed by Wieselquist, William
Browse files

Improve readme with build instructions

parent 033ba5eb
Loading
Loading
Loading
Loading
+240 −1
Original line number Diff line number Diff line
## :beers: SCALE Public Repository :beers:
# :beers: SCALE Public Repository :beers:

Welcome to the SCALE (scale.ornl.gov) public repository. The current content
is a very a sparse snapshot of the open source components in **SCALE 7.0 beta3**
@@ -6,6 +6,245 @@ that support AMPX and SAMMY, which are both open source software.

:warning: This snapshot is very experimental and subject to change. :warning:

----

## Build Instructions

### Build Prerequisites

 - git-lfs
 - C++ compiler: gcc, clang, etc. 
 - Fortran compiler: gfortran, etc.
 - hdf5 - 1.10
 - Qt5 <!-- UNTIL the next SCALE beta is pulled in here -->

### Full SCALE-public Build

To build the full SCALE-public toolset, it's recommended to install with `spack`. When building with `spack`, the main difference, **for now**, betweeen building AMPX and full SCALE-public are the build options given by CMake flags during configuration of SCALE.

### AMPX Build

There are two ways to build the AMPX toolset: with `spack` or without `spack`. Either way, start by cloning the repository:

```console
git clone https://code.ornl.gov/scale/code/scale-public.git
```

**Caveat that applies to both build methods**: Unit tests are disabled because some tests depend on packages in the full SCALE source that are not included in the public release.  

<p>
<h3>Build without spack</h3>
<details>
<summary>Details</summary>

Both the `spack` and non-`spack` builds require Trilinos. When building without `spack` you must download and build Trilinos on your own before you can build AMPX. Usually this is not a big issue.

<details>
<summary>Build Trilinos</summary>

```
git clone https://github.com/trilinos/Trilinos.git
mkdir build_trilinos
cd build_trilinos
<launch configuration script for Trilinos below>
make -j 4 install 
cd ..
```

Configuration script for Trilinos: 
```
cmake  \
-D CMAKE_Fortran_FLAGS="-fallow-argument-mismatch -fallow-invalid-boz" \
-D CMAKE_INSTALL_PREFIX:FILEPATH=${INSTALL_PATH} \
-DTrilinos_ENABLE_ALL_PACKAGES=OFF \
-DTPL_ENABLE_Boost=OFF \
-DTPL_ENABLE_Netcdf=OFF \
-DTrilinos_ENABLE_KokkosCore=ON \
-DTrilinos_ENABLE_TeuchosComm=ON \
-DTrilinos_ENABLE_TeuchosParameterList=ON \
-DTrilinos_ENABLE_TeuchosNumerics=ON \
-DTrilinos_ENABLE_Tpetra=ON \
-DTrilinos_ENABLE_Amesos2=ON \
-DTrilinos_ENABLE_Anasazi=ON \
-DTrilinos_ENABLE_Shards=ON \
-DTrilinos_ENABLE_Intrepid=ON \
-DTrilinos_ENABLE_Ifpack2=ON \
-DTrilinos_ENABLE_Stratimikos=ON \
-DBUILD_SHARED_LIBS=ON \
$*
```
</details>

Once Trilinos is built, we can build SCALE (assuming you have the other dependencies required):

```
mkdir build
cd build
<launch configuration script for SCALE below>
make -j 4 install
```

Configuration script for AMPX: 
```
cmake \
-DCMAKE_PREFIX_PATH=/path/to/trilinos/installation \
-DSCALE_USE_MCNP=OFF \
-DSCALE_USE_MPI=OFF \
-DSCALE_DEBUG=OFF \
-DSCALE_BUILD_TESTS=OFF \
-DSCALE_USE_SWIG_Python:BOOL=OFF \
-DSCALE_USE_SWIG_Fortran:BOOL=OFF \
-DCMAKE_INSTALL_PREFIX=${INSTALL_PATH} \
-DCMAKE_BUILD_TYPE=RELEASE \
$*
```

**Warning**: When running AMPX executables, this build procedure requires setting 
```bash
export LD_LIBRARY_PATH="/path/to/trilinos/installation/lib:$LD_LIBRARY_PATH"
```
at runtime.  

</details>
</p>

<p>
<h3>Build with spack</h3>
<details>
<summary>Details</summary>

To build with `spack`, first ensure you have `spack` installed. Next create a build directory and a script directory in the main `scale-public` repo:

```console
cd ~/scale-public
mkdir build
mkdir myscript
```

Next create a YAML file called `w2022.yaml` in your `myscript` directory to describe your `spack` environment and paste the following content in it:

```console
# This is the w2022-mac.yaml Spack Environment file 
spack:
  # add package specs to the `specs` list
  specs:
  - celeritas@0.1.0 cxxstd=14
  - bison@3.4.2
  - flex@2.6.4
  - 'flibcpp@1.0.1: +swig'
  - hdf5 +fortran +hl ~mpi
  - lava
  - googletest
  - gnds
  - 'ninja@1.10:'
  - nlohmann-json
  - openblas
  - openmpi@:2.999
  - pugixml
  - python +uuid
  - py-h5py@:2 ~mpi   # compound vlen bug in 2.6-3.1 and 3.3+
  - py-lxml
  - py-jsonschema
  - py-matplotlib
  - py-numpy
  - py-pandas
  - py-pint
  - py-psutil
  - py-pyyaml
  - py-sphinx
  - py-tqdm
  - silo +fortran ~mpi
  - swig@4.0.2-fortran
  - 'superlu@5:'
  - trilinos +amesos +amesos2 +anasazi +aztec +belos +ifpack gotype=long_long +epetra
    +epetraext +kokkos +ifpack2 +intrepid +ml +sacado +shards +tpetra +stratimikos
    ~mpi
  view: true
  concretizer:
    unify: true
  packages:
    celeritas:
      variants: ~geant4 ~hepmc3 ~root ~vecgeom
    hdf5:
      version: [1.10.8] # needed for h5py@2 
    hwloc:
      version: [1.11.13] # needed for openmpi 2
    qt:
      variants: ~dbus~examples~gtk~opengl~phonon+shared~sql~ssl~tools~webkit
    superlu:
      version: [5.2.2]
    trilinos:
      variants: ~adios2 ~alloptpkgs ~boost ~cgns ~chaco ~complex ~debug ~dtk ~exodus
        +explicit_template_instantiation ~float +fortran ~fortrilinos ~glm   ~gtest
        ~hdf5 ~hypre  ~intrepid2 ~isorropia ~matio ~mesquite ~metis ~minitensor ~muelu
        ~mumps ~netcdf ~nox ~openmp ~phalanx ~piro ~pnetcdf ~python ~rol ~rythmos
        +shared ~shylu ~stk ~suite-sparse ~superlu ~superlu-dist ~teko ~tempus ~x11
        ~zlib ~zoltan ~zoltan2
    all:
      providers:
        blas: [openblas]
        lapack: [openblas]
        mpi: [openmpi]
```

Following that source the environment, create and activate an environment based on the YAML file in the `scale-public` repo:

```console
. ~/spack/share/spack/setup-env.sh
spack env create w2022 w2022.yaml
spack env activate w2022
```

Then we are ready to build and compile the `spack` environment, which can take over an hour:

```console
spack concretize
spack install -j16
```

Often we've found that having Qt installed from a non-`spack` source works best, I choose `brew`:

```console
brew install qt@5
```

Next create a configuration script file called `configure-ampx.sh` in the `myscript` directory with the following inside:

```console
#!/bin/bash -x
INSTALL_PATH=${PWD}/install

#remove existing CMakeCache if it exists
if [ -e CMakeCache.txt ] ; then
   rm -v CMakeCache.txt 
   rm -fr CMakeFiles
fi

spack env activate w2022

export PATH=/usr/local/Cellar/qt@5/5.15.7/bin:$PATH

cmake \
-GNinja \
-DSCALE_USE_MCNP=OFF \
-DSCALE_USE_MPI=OFF \
-DSCALE_DEBUG=OFF \
-DSCALE_BUILD_TESTS=OFF \
-DSCALE_USE_SWIG_Python:BOOL=OFF \
-DSCALE_USE_SWIG_Fortran:BOOL=OFF \
-DCMAKE_INSTALL_PREFIX=${INSTALL_PATH} \
-DCMAKE_BUILD_TYPE=RELEASE \
$*
```

Then make and install:

```console
ninja -j16
```

</details>
</p>