README.rst 4.17 KB
Newer Older
Youngsung Kim's avatar
Youngsung Kim committed
1
2
3
4
5
6
7
8
9
10
11
12
E3SM Kernels
=================

This repository aggregates kernels extracted from the [Energy Exascale Earth System Model (E3SM)](https://e3sm.org).

Each kernel contains the requisite instructions to build and run in a standalone manner. The kernel inputs/outputs captured from the full application run would facilitate replication of kernel run to match application behavior.

Kernel locations
---------------------

Each kernels are grouped per its component. For example, all atmospheric kernels will reside under "atm" directory. You can find directories for each component at root directory.

Kim, Youngsung's avatar
Kim, Youngsung committed
13
git-lfs setup for downloading data files
Youngsung Kim's avatar
Youngsung Kim committed
14
15
---------------------------------------------------------------

Kim, Youngsung's avatar
Kim, Youngsung committed
16
**Before using kernels in this repository, you need to pull data files using lfs as explained in this section.**
Kim, Youngsung's avatar
Kim, Youngsung committed
17

Kim, Youngsung's avatar
Kim, Youngsung committed
18
If you get an error message similar to "git: 'lfs' is not a git command", it indicates that lfs is not installed.
Youngsung Kim's avatar
Youngsung Kim committed
19

Kim, Youngsung's avatar
Kim, Youngsung committed
20
21
In case that git-lfs is not available on your system, you may ask system admin to install git-lfs.
If that is not an easy option for you, you may want to install git-lfs locally by yourself.
Kim, Youngsung's avatar
Kim, Youngsung committed
22
23
24
25
26
First, download git-lfs binary from github lfs support site similar to following and copy a git-lfs
binary to a folder where path is in your $PATH environment variable. Actual path for download file
may be different from what is shown below depending on your system.:

>>> wget https://github.com/git-lfs/git-lfs/releases/download/v2.11.0/git-lfs-linux-amd64-v2.11.0.tar.gz
Kim, Youngsung's avatar
Kim, Youngsung committed
27
>>> tar -xvzf git-lfs-linux-amd64-v2.11.0.tar.gz
Kim, Youngsung's avatar
Kim, Youngsung committed
28
29
30
>>> cp git-lfs ~/bin # assuming ~/bin in your $PATH
>>> git lfs # for testing

Kim, Youngsung's avatar
Kim, Youngsung committed
31
32
You can find further explanation on using git-lfs at "https://sabicalija.github.io/git-lfs-intro/".

Kim, Youngsung's avatar
Kim, Youngsung committed
33
34
Once installed successfuly, git-lfs is ready for use in your local git repository.

Youngsung Kim's avatar
Youngsung Kim committed
35
36
37
38
39
40
41
42
43
44
Configure skipping LFS tracked files::

    >>> git lfs install --skip-smudge

Clone this repository::

    >>> git clone git@code.ornl.gov:e3sm/kernels.git

Pull single LFS tracked file::

Kim, Youngsung's avatar
Kim, Youngsung committed
45
    >>> cd kernels/atm/micro_mg_get_clols2
Youngsung Kim's avatar
Youngsung Kim committed
46
47
48
49
    >>> git lfs pull --include=micro_mg_get_cols2_0.100.0.1.dat

Pull multiple LFS tracked files::

Kim, Youngsung's avatar
Kim, Youngsung committed
50
    >>> cd kernels/atm/micro_mg_get_clols2
Youngsung Kim's avatar
Youngsung Kim committed
51
52
53
54
    >>> git lfs pull --include=micro_mg_get_cols2*

Pull all LFS tracked files in this repository::

Kim, Youngsung's avatar
Kim, Youngsung committed
55
    >>> cd kernels
Youngsung Kim's avatar
Youngsung Kim committed
56
    >>> git lfs pull
Youngsung Kim's avatar
Youngsung Kim committed
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108

How to use the kernels
------------------------

The detail usage of the kernels are explained in README.rst file in each kernel directory.

In general, Makefile in each kernel will drive usage of the kernels. For example, if you are compiling/running a kernel on the system that the kernel is extracted from, "make build" and "make run" will do. However, if you are doing on different system and/or compiler, you may need to add additional Makefile argument. Following are a couple of examples:

Cori KNL of NERSC ::

    This kernel was extracted on Cori KNL. Therefore it is simplest to use if you
    are on Cori KNL system.

    * to build : >>> make build  
    * to run : >>> make run  
    * to build with MPI : >>> make build MPI=1
    * to run with MPI : >>> make run MPI=1

Cori Haswell of NERSC ::

    To support Haswell, only PROC argument needs to be changed.

    * to build : >>> make build PROC=
    * to run : >>> make run PROC=
    * to build with MPI : >>> make build PROC= MPI=1
    * to run with MPI : >>> make run PROC= MPI=1

Summit of ORNL ::

    On Summit, IBM xlf compiler is used as default compiler.

    * to build : >>> make build COMP=xlf PROC= MACH=summit
    * to run : >>> make run COMP=xlf PROC= MACH=summit
    * to build with MPI : >>> make build COMP=xlf PROC= MACH=summit MPI=1
    * to run with MPI : >>> make run COMP=xlf PROC= MACH=summit MPI=1

Tulip of Cray Frontier-testbed ::

    On Tulip, IBM crayftn compiler is used as default compiler.

    * to build : >>> make build COMP=crayftn PROC= MACH=tulip
    * to run : >>> make run COMP=crayftn PROC= MACH=tulip
    * to build with MPI : >>> make build COMP=crayftn PROC= MACH=tulip MPI=1
    * to run with MPI : >>> make run COMP=crayftn PROC= MACH=tulip MPI=1

Generic Linux::

    Gfortran is freely available for many Linux distros.

    * to build : >>> make build COMP=gfortran PROC=
    * to run : >>> make run COMP=gfortran PROC=