README.rst 4.14 KB
Newer Older
Youngsung Kim's avatar
Youngsung Kim committed
1
2
3
4
5
6
7
8
9
10
11
12
E3SM Kernels
=================

This repository aggregates kernels extracted from the [Energy Exascale Earth System Model (E3SM)](https://e3sm.org).

Each kernel contains the requisite instructions to build and run in a standalone manner. The kernel inputs/outputs captured from the full application run would facilitate replication of kernel run to match application behavior.

Kernel locations
---------------------

Each kernels are grouped per its component. For example, all atmospheric kernels will reside under "atm" directory. You can find directories for each component at root directory.

Youngsung Kim's avatar
Youngsung Kim committed
13
14
15
How to use git-lfs to prevent using unnecessary large files
---------------------------------------------------------------

Kim, Youngsung's avatar
Kim, Youngsung committed
16
**Before using kernels in this repository, you need to pull data files using lfs as explained in this section.**
Kim, Youngsung's avatar
Kim, Youngsung committed
17
18

You can find further explanation on using git-lfs at "https://sabicalija.github.io/git-lfs-intro/".
Kim, Youngsung's avatar
Kim, Youngsung committed
19
If you get an error message similar to "git: 'lfs' is not a git command", please install lfs first as explaiend at above link
Youngsung Kim's avatar
Youngsung Kim committed
20

Kim, Youngsung's avatar
Kim, Youngsung committed
21
22
23
24
25
26
27
28
29
30
31
32
33
In case that git-lfs is not available on your system, you may want to install git-lfs locally.
First, download git-lfs binary from github lfs support site similar to following and copy a git-lfs
binary to a folder where path is in your $PATH environment variable. Actual path for download file
may be different from what is shown below depending on your system.:

>>> wget https://github.com/git-lfs/git-lfs/releases/download/v2.11.0/git-lfs-linux-amd64-v2.11.0.tar.gz
>>> gunzip git-lfs-linux-amd64-v2.11.0.tar.gz
>>> tar -xvf git-lfs-linux-amd64-v2.11.0.tar
>>> cp git-lfs ~/bin # assuming ~/bin in your $PATH
>>> git lfs # for testing

Once installed successfuly, git-lfs is ready for use in your local git repository.

Youngsung Kim's avatar
Youngsung Kim committed
34
35
36
37
38
39
40
41
42
43
Configure skipping LFS tracked files::

    >>> git lfs install --skip-smudge

Clone this repository::

    >>> git clone git@code.ornl.gov:e3sm/kernels.git

Pull single LFS tracked file::

Kim, Youngsung's avatar
Kim, Youngsung committed
44
    >>> cd kernels/atm/micro_mg_get_clols2
Youngsung Kim's avatar
Youngsung Kim committed
45
46
47
48
    >>> git lfs pull --include=micro_mg_get_cols2_0.100.0.1.dat

Pull multiple LFS tracked files::

Kim, Youngsung's avatar
Kim, Youngsung committed
49
    >>> cd kernels/atm/micro_mg_get_clols2
Youngsung Kim's avatar
Youngsung Kim committed
50
51
52
53
    >>> git lfs pull --include=micro_mg_get_cols2*

Pull all LFS tracked files in this repository::

Kim, Youngsung's avatar
Kim, Youngsung committed
54
    >>> cd kernels
Youngsung Kim's avatar
Youngsung Kim committed
55
    >>> git lfs pull
Youngsung Kim's avatar
Youngsung Kim committed
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107

How to use the kernels
------------------------

The detail usage of the kernels are explained in README.rst file in each kernel directory.

In general, Makefile in each kernel will drive usage of the kernels. For example, if you are compiling/running a kernel on the system that the kernel is extracted from, "make build" and "make run" will do. However, if you are doing on different system and/or compiler, you may need to add additional Makefile argument. Following are a couple of examples:

Cori KNL of NERSC ::

    This kernel was extracted on Cori KNL. Therefore it is simplest to use if you
    are on Cori KNL system.

    * to build : >>> make build  
    * to run : >>> make run  
    * to build with MPI : >>> make build MPI=1
    * to run with MPI : >>> make run MPI=1

Cori Haswell of NERSC ::

    To support Haswell, only PROC argument needs to be changed.

    * to build : >>> make build PROC=
    * to run : >>> make run PROC=
    * to build with MPI : >>> make build PROC= MPI=1
    * to run with MPI : >>> make run PROC= MPI=1

Summit of ORNL ::

    On Summit, IBM xlf compiler is used as default compiler.

    * to build : >>> make build COMP=xlf PROC= MACH=summit
    * to run : >>> make run COMP=xlf PROC= MACH=summit
    * to build with MPI : >>> make build COMP=xlf PROC= MACH=summit MPI=1
    * to run with MPI : >>> make run COMP=xlf PROC= MACH=summit MPI=1

Tulip of Cray Frontier-testbed ::

    On Tulip, IBM crayftn compiler is used as default compiler.

    * to build : >>> make build COMP=crayftn PROC= MACH=tulip
    * to run : >>> make run COMP=crayftn PROC= MACH=tulip
    * to build with MPI : >>> make build COMP=crayftn PROC= MACH=tulip MPI=1
    * to run with MPI : >>> make run COMP=crayftn PROC= MACH=tulip MPI=1

Generic Linux::

    Gfortran is freely available for many Linux distros.

    * to build : >>> make build COMP=gfortran PROC=
    * to run : >>> make run COMP=gfortran PROC=