README.rst 4.35 KB
Newer Older
Youngsung Kim's avatar
Youngsung Kim committed
1
2
3
4
5
6
7
8
9
10
11
12
E3SM Kernels
=================

This repository aggregates kernels extracted from the [Energy Exascale Earth System Model (E3SM)](https://e3sm.org).

Each kernel contains the requisite instructions to build and run in a standalone manner. The kernel inputs/outputs captured from the full application run would facilitate replication of kernel run to match application behavior.

Kernel locations
---------------------

Each kernels are grouped per its component. For example, all atmospheric kernels will reside under "atm" directory. You can find directories for each component at root directory.

Kim, Youngsung's avatar
Kim, Youngsung committed
13
git-lfs client setup for downloading data files
Youngsung Kim's avatar
Youngsung Kim committed
14
15
---------------------------------------------------------------

Kim, Youngsung's avatar
Kim, Youngsung committed
16
**Before using kernels in this repository, you need to pull data files using lfs as explained in this section.**
Kim, Youngsung's avatar
Kim, Youngsung committed
17

Kim, Youngsung's avatar
Kim, Youngsung committed
18
If you get an error message similar to "git: 'lfs' is not a git command", it indicates that lfs is not installed.
Youngsung Kim's avatar
Youngsung Kim committed
19

Kim, Youngsung's avatar
Kim, Youngsung committed
20
21
22
In case that git-lfs client is not available on your system, you may ask system admin to install git-lfs client.
If that is not an easy option for you, you may want to install git-lfs client locally by yourself.
First, download git-lfs client binary from github lfs support site(https://github.com/git-lfs/git-lfs/releases)
Kim, Youngsung's avatar
Kim, Youngsung committed
23
24
similar to following and copy a git-lfs client binary to a folder whose path is in your $PATH environment variable.
Actual url to git-lfs release may be different from what is shown below depending on your system.
Kim, Youngsung's avatar
Kim, Youngsung committed
25

Kim, Youngsung's avatar
Kim, Youngsung committed
26
Example for linux-amd64 compatible system:
Kim, Youngsung's avatar
Kim, Youngsung committed
27
28

>>> wget https://github.com/git-lfs/git-lfs/releases/download/v2.11.0/git-lfs-linux-amd64-v2.11.0.tar.gz
Kim, Youngsung's avatar
Kim, Youngsung committed
29
>>> tar -xvzf git-lfs-linux-amd64-v2.11.0.tar.gz
Kim, Youngsung's avatar
Kim, Youngsung committed
30
31
32
>>> cp git-lfs ~/bin # assuming ~/bin in your $PATH
>>> git lfs # for testing

Kim, Youngsung's avatar
Kim, Youngsung committed
33
34
You can find further explanation on using git-lfs at "https://sabicalija.github.io/git-lfs-intro/".

Kim, Youngsung's avatar
Kim, Youngsung committed
35
Once installed successfuly, git-lfs client is ready for use in your local git repository.
Kim, Youngsung's avatar
Kim, Youngsung committed
36

Youngsung Kim's avatar
Youngsung Kim committed
37
38
39
40

Clone this repository::

    >>> git clone git@code.ornl.gov:e3sm/kernels.git
Kim, Youngsung's avatar
Kim, Youngsung committed
41
42
43
44
45
    >>> cd kernels
    
Install git-lfs on this repo. with skipping LFS tracked files::

    >>> git lfs install --skip-smudge
Youngsung Kim's avatar
Youngsung Kim committed
46
47
48

Pull single LFS tracked file::

Kim, Youngsung's avatar
Kim, Youngsung committed
49
    >>> cd kernels/atm/micro_mg_get_clols2
Youngsung Kim's avatar
Youngsung Kim committed
50
51
52
53
    >>> git lfs pull --include=micro_mg_get_cols2_0.100.0.1.dat

Pull multiple LFS tracked files::

Kim, Youngsung's avatar
Kim, Youngsung committed
54
    >>> cd kernels/atm/micro_mg_get_clols2
Youngsung Kim's avatar
Youngsung Kim committed
55
56
57
58
    >>> git lfs pull --include=micro_mg_get_cols2*

Pull all LFS tracked files in this repository::

Kim, Youngsung's avatar
Kim, Youngsung committed
59
    >>> cd kernels
Youngsung Kim's avatar
Youngsung Kim committed
60
    >>> git lfs pull
Youngsung Kim's avatar
Youngsung Kim committed
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112

How to use the kernels
------------------------

The detail usage of the kernels are explained in README.rst file in each kernel directory.

In general, Makefile in each kernel will drive usage of the kernels. For example, if you are compiling/running a kernel on the system that the kernel is extracted from, "make build" and "make run" will do. However, if you are doing on different system and/or compiler, you may need to add additional Makefile argument. Following are a couple of examples:

Cori KNL of NERSC ::

    This kernel was extracted on Cori KNL. Therefore it is simplest to use if you
    are on Cori KNL system.

    * to build : >>> make build  
    * to run : >>> make run  
    * to build with MPI : >>> make build MPI=1
    * to run with MPI : >>> make run MPI=1

Cori Haswell of NERSC ::

    To support Haswell, only PROC argument needs to be changed.

    * to build : >>> make build PROC=
    * to run : >>> make run PROC=
    * to build with MPI : >>> make build PROC= MPI=1
    * to run with MPI : >>> make run PROC= MPI=1

Summit of ORNL ::

    On Summit, IBM xlf compiler is used as default compiler.

    * to build : >>> make build COMP=xlf PROC= MACH=summit
    * to run : >>> make run COMP=xlf PROC= MACH=summit
    * to build with MPI : >>> make build COMP=xlf PROC= MACH=summit MPI=1
    * to run with MPI : >>> make run COMP=xlf PROC= MACH=summit MPI=1

Tulip of Cray Frontier-testbed ::

    On Tulip, IBM crayftn compiler is used as default compiler.

    * to build : >>> make build COMP=crayftn PROC= MACH=tulip
    * to run : >>> make run COMP=crayftn PROC= MACH=tulip
    * to build with MPI : >>> make build COMP=crayftn PROC= MACH=tulip MPI=1
    * to run with MPI : >>> make run COMP=crayftn PROC= MACH=tulip MPI=1

Generic Linux::

    Gfortran is freely available for many Linux distros.

    * to build : >>> make build COMP=gfortran PROC=
    * to run : >>> make run COMP=gfortran PROC=