ADIOS2 issueshttps://code.ornl.gov/pnb/ADIOS2/-/issues2020-03-02T12:01:51Zhttps://code.ornl.gov/pnb/ADIOS2/-/issues/1906Reading a file with many subfiles fails2020-03-02T12:01:51ZPodhorszki, NorbertReading a file with many subfiles fails*Created by: pnorbert*
I thought this was addressed before but I don't find an issue for it.
I have an XGC output with 1024 subfiles on a KNL machine. Trying to bpls dump a variable that is written from all processes fails when openin...*Created by: pnorbert*
I thought this was addressed before but I don't find an issue for it.
I have an XGC output with 1024 subfiles on a KNL machine. Trying to bpls dump a variable that is written from all processes fails when opening too many subfiles.
```
$ bpls -l xgc.restart.00006.bp/ -d inum_arr -n 16
int32_t inum_arr {1024} = 2293 / 4061532
bpls caught an exception
ERROR: couldn't open file xgc.restart.00006.bp/data.1019, check permissions or path existence, in call to POSIX open
$ ls -l xgc.restart.00006.bp/data.1019
-rw-r----- 1 n55002 gn55 321625842 Dec 18 11:50 xgc.restart.00006.bp/data.1019
```
`$ bp4dbg.py xgc.restart.00006.bp/data.1019` dumps a perfectly fine data file.
https://code.ornl.gov/pnb/ADIOS2/-/issues/1665Reading Fortran data using python interface2019-08-09T21:33:40ZPodhorszki, NorbertReading Fortran data using python interface*Created by: germasch*
When reading data written by a Fortran code using python, indices end up being reversed:
Sample Fortran code (it's based on the included heatmap test)
```fortran
program TestBPWriteReadHeatMap2D
use mpi
...*Created by: germasch*
When reading data written by a Fortran code using python, indices end up being reversed:
Sample Fortran code (it's based on the included heatmap test)
```fortran
program TestBPWriteReadHeatMap2D
use mpi
use adios2
implicit none
type(adios2_adios) :: adios
type(adios2_io) :: ioPut
type(adios2_engine) :: bpWriter
type(adios2_variable) :: var
real(kind=8) :: temperature(0:4, 0:9)
integer(kind=8), parameter :: ishape(2) = (/5, 10/)
integer(kind=8), parameter :: istart(2) = (/0, 0/)
integer :: ierr
call mpi_init(ierr)
call adios2_init(adios, MPI_COMM_WORLD, adios2_debug_mode_on, ierr)
call random_number(temperature)
print*, 'sample temperature(2,8) =', temperature(2, 8)
call adios2_declare_io(ioPut, adios, 'HeatMapWrite', ierr)
call adios2_define_variable(var, ioPut, 'temperature', adios2_type_dp, &
2, ishape, istart, ishape, &
adios2_constant_dims, ierr)
call adios2_open(bpWriter, ioPut, 'HeatMap2D_f.bp', adios2_mode_write, &
ierr)
call adios2_put(bpWriter, var, temperature, ierr)
call adios2_close(bpWriter, ierr)
call adios2_finalize(adios, ierr)
call mpi_finalize(ierr)
end program TestBPWriteReadHeatMap2D
```
Output:
```sh
[kai@macbook build-fortran (pr/fix-mgard *)]$ bin/test_fortran
sample temperature(2,8) = 0.13062623044039723
```
Reading the bp file from Python:
```
import adios2
fh = adios2.open("/Users/kai/build/ADIOS2/build-fortran/HeatMap2D_f.bp", "r")
t = fh.read('temperature')
print(t[2, 8])
```
gives
```
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-10-cf0fcf0017a8> in <module>
2 fh = adios2.open("/Users/kai/build/ADIOS2/build-fortran/HeatMap2D_f.bp", "r")
3 t = fh.read('temperature')
----> 4 print(t[2, 8])
IndexError: index 8 is out of bounds for axis 1 with size 5
```
To access the Fortran data element `t(2, 8)` from Python, one has to reverse the index order:
```python
>>> print(t[8, 2])
0.13062623044039723
```
This is related to #1661 , but actually fixable without API additions.https://code.ornl.gov/pnb/ADIOS2/-/issues/2033segfault running adios_reorganize to convert to hdf52020-03-11T17:02:51ZPodhorszki, Norbertsegfault running adios_reorganize to convert to hdf5*Created by: jhidding*
**Describe the bug**
Running the example line: `adios_reorganize sim.bp sim.h5 BPFile "" HDF5 ""` from the documentation creates a segmentation fault. Both with and without the `mpirun` bit.
**To Reproduce**
...*Created by: jhidding*
**Describe the bug**
Running the example line: `adios_reorganize sim.bp sim.h5 BPFile "" HDF5 ""` from the documentation creates a segmentation fault. Both with and without the `mpirun` bit.
**To Reproduce**
1. have a .bp file (in this case created with the OpenFOAM Adios writer)
2. run `adios_reorganize`, doesn't matter which output back-end is used
**Expected behavior**
no segfault
**Desktop (please complete the following information):**
- OS/Platform: Fedora 31
- Build: gcc 9.2.1
**Additional context**
The same bp file seems to read fine using the Python module.
printed backtrace:
```
[ESLT0107:12052:0:12052] Caught signal 11 (Segmentation fault: address not mapped to object at address (nil))
==== backtrace ====
0 /lib64/libucs.so.0(+0x1b25f) [0x7f40e808525f]
1 /lib64/libucs.so.0(+0x1b42a) [0x7f40e808542a]
2 /lib64/libc.so.6(cfree+0x52) [0x7f40ea63b7e2]
3 /home/johannes/.local/lib/libadios2.so.2(_ZN6adios26format15BP3Deserializer12PostDataReadIdEEvRNS_4core8VariableIT_EERNS6_4InfoERKNS_6helper16SubStreamBoxInfoEbm+0x265) [0x7f40eb02e945]
4 /home/johannes/.local/lib/libadios2.so.2(_ZN6adios24core6engine9BP3Reader18ReadVariableBlocksIdEEvRNS0_8VariableIT_EE+0x19e) [0x7f40eaf3e84e]
5 /home/johannes/.local/lib/libadios2.so.2(_ZN6adios24core6engine9BP3Reader9DoGetSyncERNS0_8VariableIdEEPd+0x86) [0x7f40eaf378a6]
6 /home/johannes/.local/lib/libadios2.so.2(_ZN6adios24core6Engine3GetIdEEvRNS0_8VariableIT_EEPS4_NS_4ModeE+0x19b) [0x7f40eae9215b]
7 /home/johannes/.local/lib/libadios2.so.2(_ZN6adios24core6Engine3GetIdEEvRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPT_NS_4ModeE+0x64) [0x7f40eae922e4]
8 adios_reorganize() [0x409232]
9 adios_reorganize() [0x40cfd7]
10 adios_reorganize() [0x405f2a]
11 /lib64/libc.so.6(__libc_start_main+0xf3) [0x7f40ea5d61a3]
12 adios_reorganize() [0x4061be]
===================
```
**follow up**
Let me know if you need me to repeat this with ADIOS2 built with build-type set to Debughttps://code.ornl.gov/pnb/ADIOS2/-/issues/1728SST crashes with LibFabric2019-09-28T18:21:38ZPodhorszki, NorbertSST crashes with LibFabric*Created by: khuck*
ADIOS2 was configured on a 36-core Linux workstation with the following cmake output:
```
Currently Loaded Modules:
1) gcc/8.1 2) mpi/openmpi-4.0.1_gcc-8.1 3) cmake/3.15.1 4) python/3.6.8
++ which m...*Created by: khuck*
ADIOS2 was configured on a 36-core Linux workstation with the following cmake output:
```
Currently Loaded Modules:
1) gcc/8.1 2) mpi/openmpi-4.0.1_gcc-8.1 3) cmake/3.15.1 4) python/3.6.8
++ which mpicc
++ which mpic++
++ which mpif90
+ cmake -DCMAKE_C_COMPILER=/packages/openmpi/4.0.1-gcc8.1/bin/mpicc -DCMAKE_CXX_COMPILER=/packages/openmpi/4.0.1-gcc8.1/bin/mpic++ -DCMAKE_Fortran_COMPILER=/packages/openmpi/4.0.1-gcc8.1/bin/mpif90 -DADIOS2_USE_Python=ON -DCMAKE_INSTALL_PREFIX=/home/users/khuck/src/ADIOS2/install_mpi -DCMAKE_BUILD_TYPE=RelWithDebInfo ..
-- The C compiler identification is GNU 8.1.0
-- The CXX compiler identification is GNU 8.1.0
-- Check for working C compiler: /packages/openmpi/4.0.1-gcc8.1/bin/mpicc
-- Check for working C compiler: /packages/openmpi/4.0.1-gcc8.1/bin/mpicc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /packages/openmpi/4.0.1-gcc8.1/bin/mpic++
-- Check for working CXX compiler: /packages/openmpi/4.0.1-gcc8.1/bin/mpic++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Could NOT find Blosc (missing: BLOSC_LIBRARY BLOSC_INCLUDE_DIR)
-- Could NOT find BZip2 (missing: BZIP2_LIBRARIES BZIP2_INCLUDE_DIR)
-- Could NOT find ZFP (missing: ZFP_LIBRARY ZFP_INCLUDE_DIR)
-- Could NOT find SZ (missing: SZ_LIBRARY ZLIB_LIBRARY ZSTD_LIBRARY SZ_INCLUDE_DIR)
-- Could NOT find MGARD (missing: MGARD_LIBRARY ZLIB_LIBRARY MGARD_INCLUDE_DIR)
-- Found ZLIB: /usr/lib64/libz.so (found version "1.2.7")
-- Could NOT find PNG: Found unsuitable version "1.5.13", but required is at least "1.6.0" (found /usr/lib64/libpng.so)
-- The Fortran compiler identification is GNU 8.1.0
-- Check for working Fortran compiler: /packages/openmpi/4.0.1-gcc8.1/bin/mpif90
-- Check for working Fortran compiler: /packages/openmpi/4.0.1-gcc8.1/bin/mpif90 -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether /packages/openmpi/4.0.1-gcc8.1/bin/mpif90 supports Fortran 90
-- Checking whether /packages/openmpi/4.0.1-gcc8.1/bin/mpif90 supports Fortran 90 -- yes
-- Found MPI_C: /packages/openmpi/4.0.1-gcc8.1/bin/mpicc (found version "3.1")
-- Found MPI_CXX: /packages/openmpi/4.0.1-gcc8.1/bin/mpic++ (found version "3.1")
-- Found MPI_Fortran: /packages/openmpi/4.0.1-gcc8.1/bin/mpif90 (found version "3.1")
-- Found MPI: TRUE (found version "3.1") found components: C Fortran CXX
-- Found ZeroMQ: /usr/lib64/libzmq.so (found suitable version "4.1.4", minimum required is "4.1")
-- Could NOT find HDF5 (missing: HDF5_LIBRARIES HDF5_INCLUDE_DIRS C) (found version "")
-- Found PythonInterp: /packages/python/3.6.8/bin/python3 (found version "3.6.8")
-- Found PythonLibs: /packages/python/3.6.8/lib/libpython3.6m.so (found version "3.6.8")
-- Found PythonModule_numpy: /packages/python/3.6.8/lib/python3.6/site-packages/numpy
-- Found PythonModule_mpi4py: /home/users/khuck/.local/lib/python3.6/site-packages/mpi4py
-- Found PythonFull: /packages/python/3.6.8/bin/python3 found components: Interp Libs numpy mpi4py
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.27.1")
-- Checking for module 'libfabric'
-- Found libfabric, version 1.6.1
-- Found LIBFABRIC: /usr/lib64/libfabric.so (Required is at least version "1.6")
-- Checking for module 'cray-drc'
-- No package 'cray-drc' found
-- Could NOT find CrayDRC (missing: CrayDRC_LIBRARIES)
-- Looking for shmget
-- Looking for shmget - found
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- ADIOS2 ThirdParty: Configuring KWSys
-- Checking whether header cstdio is available
-- Checking whether header cstdio is available - yes
-- Checking for Large File Support
-- Checking for Large File Support - yes
-- Checking whether C++ compiler has 'long long'
-- Checking whether C++ compiler has 'long long' - yes
-- Checking whether C++ compiler has '__int64'
-- Checking whether C++ compiler has '__int64' - no
-- Checking whether wstring is available
-- Checking whether wstring is available - yes
-- Checking whether C compiler has ptrdiff_t in stddef.h
-- Checking whether C compiler has ptrdiff_t in stddef.h - yes
-- Checking whether C compiler has ssize_t in unistd.h
-- Checking whether C compiler has ssize_t in unistd.h - yes
-- Checking whether CXX compiler has setenv
-- Checking whether CXX compiler has setenv - yes
-- Checking whether CXX compiler has unsetenv
-- Checking whether CXX compiler has unsetenv - yes
-- Checking whether CXX compiler has environ in stdlib.h
-- Checking whether CXX compiler has environ in stdlib.h - no
-- Checking whether CXX compiler has utimes
-- Checking whether CXX compiler has utimes - yes
-- Checking whether CXX compiler has utimensat
-- Checking whether CXX compiler has utimensat - yes
-- Checking whether CXX compiler struct stat has st_mtim member
-- Checking whether CXX compiler struct stat has st_mtim member - yes
-- Checking whether CXX compiler struct stat has st_mtimespec member
-- Checking whether CXX compiler struct stat has st_mtimespec member - no
-- Checking whether <ext/stdio_filebuf.h> is available
-- Checking whether <ext/stdio_filebuf.h> is available - yes
-- ADIOS2 ThirdParty: Configuring GTest
-- ADIOS2 ThirdParty: Configuring pybind11
-- Found PythonLibs: /packages/python/3.6.8/lib/libpython3.6m.so
-- pybind11 v2.2.4
-- ADIOS2 ThirdParty: Configuring pugixml
-- ADIOS2 ThirdParty: Configuring nlohmann_json
-- ADIOS2 ThirdParty: Configuring atl
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of double
-- Check size of double - done
-- Check size of float
-- Check size of float - done
-- Check size of int
-- Check size of int - done
-- Check size of short
-- Check size of short - done
-- Looking for include file malloc.h
-- Looking for include file malloc.h - found
-- Looking for include file unistd.h
-- Looking for include file unistd.h - found
-- Looking for include file stdlib.h
-- Looking for include file stdlib.h - found
-- Looking for include file string.h
-- Looking for include file string.h - found
-- Looking for include file sys/time.h
-- Looking for include file sys/time.h - found
-- Looking for include file windows.h
-- Looking for include file windows.h - not found
-- Looking for fork
-- Looking for fork - found
-- Found atl: /home/users/khuck/src/ADIOS2/build/thirdparty/atl/atl/atl-config.cmake (found version "2.2.1")
-- ADIOS2 ThirdParty: Configuring dill
-- Check size of void*
-- Check size of void* - done
-- Check size of long
-- Check size of long - done
-- Check if the system is big endian
-- Searching 16 bit integer
-- Check size of unsigned short
-- Check size of unsigned short - done
-- Using unsigned short
-- Check if the system is big endian - little endian
-- Checking for module 'libffi'
-- Found libffi, version 3.0.13
-- Found LibFFI: -lffi
-- Enabling emulation
-- Looking for include file stdarg.h
-- Looking for include file stdarg.h - found
-- Looking for include file memory.h
-- Looking for include file memory.h - found
-- Found dill: /home/users/khuck/src/ADIOS2/build/thirdparty/dill/dill/dill-config.cmake (found version "2.4.0")
-- ADIOS2 ThirdParty: Configuring ffs
-- Check size of off_t
-- Check size of off_t - done
-- Check size of long double
-- Check size of long double - done
-- Check size of long long
-- Check size of long long - done
-- Check size of size_t
-- Check size of size_t - done
-- Looking for socket
-- Looking for socket - found
-- Found BISON: /usr/bin/bison (found version "3.0.4")
-- Found FLEX: /usr/bin/flex (found version "2.5.37")
-- Found dill: /home/users/khuck/src/ADIOS2/build/thirdparty/dill/dill/dill-config.cmake (found suitable version "2.4.0", minimum required is "2.3.1")
-- Found atl: /home/users/khuck/src/ADIOS2/build/thirdparty/atl/atl/atl-config.cmake (found suitable version "2.2.1", minimum required is "2.2.1")
-- Looking for netdb.h
-- Looking for netdb.h - found
-- Looking for sockLib.h
-- Looking for sockLib.h - not found
-- Looking for sys/select.h
-- Looking for sys/select.h - found
-- Looking for sys/socket.h
-- Looking for sys/socket.h - found
-- Looking for sys/times.h
-- Looking for sys/times.h - found
-- Looking for sys/uio.h
-- Looking for sys/uio.h - found
-- Looking for sys/un.h
-- Looking for sys/un.h - found
-- Looking for winsock.h
-- Looking for winsock.h - not found
-- Looking for strtof
-- Looking for strtof - found
-- Looking for strtod
-- Looking for strtod - found
-- Looking for strtold
-- Looking for strtold - found
-- Looking for getdomainname
-- Looking for getdomainname - found
-- Check size of struct iovec
-- Check size of struct iovec - done
-- Performing Test HAS_IOV_BASE_IOVEC
-- Performing Test HAS_IOV_BASE_IOVEC - Success
-- Found atl: /home/users/khuck/src/ADIOS2/build/thirdparty/atl/atl/atl-config.cmake (found version "2.2.1")
-- Found ffs: /home/users/khuck/src/ADIOS2/build/thirdparty/ffs/ffs/ffs-config.cmake (found version "1.6.0")
-- ADIOS2 ThirdParty: Configuring enet
-- Looking for getaddrinfo
-- Looking for getaddrinfo - found
-- Looking for getnameinfo
-- Looking for getnameinfo - found
-- Looking for gethostbyaddr_r
-- Looking for gethostbyaddr_r - found
-- Looking for gethostbyname_r
-- Looking for gethostbyname_r - found
-- Looking for poll
-- Looking for poll - found
-- Looking for fcntl
-- Looking for fcntl - found
-- Looking for inet_pton
-- Looking for inet_pton - found
-- Looking for inet_ntop
-- Looking for inet_ntop - found
-- Performing Test HAS_MSGHDR_FLAGS
-- Performing Test HAS_MSGHDR_FLAGS - Success
-- Performing Test HAS_SOCKLEN_T
-- Performing Test HAS_SOCKLEN_T - Success
-- Found enet: /home/users/khuck/src/ADIOS2/build/thirdparty/enet/enet/enet-config.cmake (found version "1.3.14")
-- ADIOS2 ThirdParty: Configuring EVPath
-- Performing Test HAVE_MATH
-- Performing Test HAVE_MATH - Failed
-- Performing Test HAVE_LIBM_MATH
-- Performing Test HAVE_LIBM_MATH - Success
-- Found atl: /home/users/khuck/src/ADIOS2/build/thirdparty/atl/atl/atl-config.cmake (found suitable version "2.2.1", minimum required is "2.2.1")
-- Found atl: /home/users/khuck/src/ADIOS2/build/thirdparty/atl/atl/atl-config.cmake (found version "2.2.1")
-- Found ffs: /home/users/khuck/src/ADIOS2/build/thirdparty/ffs/ffs/ffs-config.cmake (found suitable version "1.6.0", minimum required is "1.5.1")
-- Could NOT find nvml (missing: NVML_INCLUDE_DIR)
-- Looking for clock_gettime
-- Looking for clock_gettime - found
-- Found enet: /home/users/khuck/src/ADIOS2/build/thirdparty/enet/enet/enet-config.cmake (found suitable version "1.3.14", minimum required is "1.3.13")
-- - Udt4 library was not found. This is not a fatal error, just that the Udt4 transport will not be built.
-- Found LIBFABRIC: /usr/lib64/libfabric.so
-- Looking for ibv_create_qp
-- Looking for ibv_create_qp - not found
-- Looking for ibv_create_qp in ibverbs
-- Looking for ibv_create_qp in ibverbs - found
-- Found IBVERBS: ibverbs
-- Could NOT find nnti (missing: NNTI_INCLUDE_DIR NNTI_trios_nnti_LIBRARY NNTI_trios_support_LIBRARY)
-- Looking for hostlib.h
-- Looking for hostlib.h - not found
-- Looking for sys/sockio.h
-- Looking for sys/sockio.h - not found
-- Performing Test HAVE_FDS_BITS
-- Performing Test HAVE_FDS_BITS - Failed
-- Looking for writev
-- Looking for writev - found
-- Looking for uname
-- Looking for uname - found
-- Looking for getloadavg
-- Looking for getloadavg - found
-- Looking for gettimeofday
-- Looking for gettimeofday - found
-- Looking for getifaddrs
-- Looking for getifaddrs - found
-- Found atl: /home/users/khuck/src/ADIOS2/build/thirdparty/atl/atl/atl-config.cmake (found suitable version "2.2.1", minimum required is "2.2.1")
-- Found atl: /home/users/khuck/src/ADIOS2/build/thirdparty/atl/atl/atl-config.cmake (found version "2.2.1")
-- Found ffs: /home/users/khuck/src/ADIOS2/build/thirdparty/ffs/ffs/ffs-config.cmake (found suitable version "1.6.0", minimum required is "1.6.0")
-- Found EVPath: /home/users/khuck/src/ADIOS2/build/thirdparty/EVPath/EVPath/EVPathConfigCommon.cmake (found version "4.4.0")
-- Found atl: /home/users/khuck/src/ADIOS2/build/thirdparty/atl/atl/atl-config.cmake (found suitable version "2.2.1", minimum required is "2.2.1")
-- Found atl: /home/users/khuck/src/ADIOS2/build/thirdparty/atl/atl/atl-config.cmake (found version "2.2.1")
-- Looking for rdma/fi_ext_gni.h
-- Looking for rdma/fi_ext_gni.h - not found
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- LTO enabled
-- Detecting Fortran/C Interface
-- Detecting Fortran/C Interface - Found GLOBAL and MODULE mangling
-- Verifying Fortran/CXX Compiler Compatibility
-- Verifying Fortran/CXX Compiler Compatibility - Success
-- Found MPI: TRUE (found version "3.1") found components: C
ADIOS2 build configuration:
ADIOS Version: 2.4.0
C++ Compiler : GNU 8.1.0
/packages/openmpi/4.0.1-gcc8.1/bin/mpic++
Fortran Compiler : GNU 8.1.0
/packages/openmpi/4.0.1-gcc8.1/bin/mpif90
Installation prefix: /home/users/khuck/src/ADIOS2/install_mpi
bin: bin
lib: lib64
include: include
cmake: lib64/cmake/adios2
python: lib64/python3.6/site-packages
Features:
Library Type: shared
Build Type: RelWithDebInfo
Testing: ON
Build Options:
Blosc : OFF
BZip2 : OFF
ZFP : OFF
SZ : OFF
MGARD : OFF
PNG : OFF
MPI : ON
DataMan : ON
SSC : ON
SST : ON
ZeroMQ : ON
HDF5 : OFF
Python : ON
Fortran : ON
SysVShMem: ON
Profiling: ON
Endian_Reverse: OFF
RDMA Transport for Staging: Available
-- Configuring done
-- Generating done
-- Build files have been written to: /home/users/khuck/src/ADIOS2/build
```
When trying to run the heatTransfer example on this workstation with SST, the following crash happened (similar/same crash happens without the --mca arguments):
```
mpirun --mca btl_openib_allow_ib true --mca btl_openib_warn_default_gid_prefix 0 -n 16 ./heatSimulation sim.bp 4 4 64 64 100 10 : -n 4 ./heatAnalysis sim.bp analysis.bp 2 2
Process decomposition : 4 x 4
Array size per process : 64 x 64
Number of output steps : 100
Iterations per step : 10
Using SST engine for input
Using BP4 engine for output
[delphi:53491] *** Process received signal ***
[delphi:53491] Signal: Segmentation fault (11)
[delphi:53491] Signal code: Address not mapped (1)
[delphi:53491] Failing at address: 0x10
[delphi:53491] [ 0] /lib64/libpthread.so.0(+0xf5d0)[0x7f2fc755c5d0]
[delphi:53491] [ 1] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(+0x17080)[0x7f2fc6d00080]
[delphi:53491] [ 2] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(+0x171ec)[0x7f2fc6d001ec]
[delphi:53491] [ 3] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(SstWriterOpen+0xc6)[0x7f2fc6cf6ba6]
[delphi:53491] [ 4] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios24core6engine9SstWriterC2ERNS0_2IOERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0x167)[0x7f2fc8a14827]
[delphi:53491] [ 5] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios24core2IO4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0x5f3)[0x7f2fc871b003]
[delphi:53491] [ 6] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios22IO4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0xe3)[0x7f2fc8a7c3d3]
[delphi:53491] [ 7] ./heatSimulation[0x40fb36]
[delphi:53491] [ 8] ./heatSimulation[0x40b4ef]
[delphi:53491] [ 9] /lib64/libc.so.6(__libc_start_main+0xf5)[0x7f2fc71a23d5]
[delphi:53491] [10] ./heatSimulation[0x40b71f]
[delphi:53491] *** End of error message ***
[delphi:53489] *** Process received signal ***
[delphi:53489] Signal: Segmentation fault (11)
[delphi:53489] Signal code: Address not mapped (1)
[delphi:53489] Failing at address: 0x10
[delphi:53489] [ 0] /lib64/libpthread.so.0(+0xf5d0)[0x7f115d8045d0]
[delphi:53489] [ 1] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(+0x17080)[0x7f115cfa8080]
[delphi:53489] [ 2] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(+0x171ec)[0x7f115cfa81ec]
[delphi:53489] [ 3] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(SstWriterOpen+0xc6)[0x7f115cf9eba6]
[delphi:53489] [ 4] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios24core6engine9SstWriterC2ERNS0_2IOERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0x167)[0x7f115ecbc827]
[delphi:53489] [ 5] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios24core2IO4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0x5f3)[0x7f115e9c3003]
[delphi:53489] [ 6] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios22IO4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0xe3)[0x7f115ed243d3]
[delphi:53489] [ 7] ./heatSimulation[0x40fb36]
[delphi:53489] [ 8] ./heatSimulation[0x40b4ef]
[delphi:53489] [ 9] /lib64/libc.so.6(__libc_start_main+0xf5)[0x7f115d44a3d5]
[delphi:53489] [10] ./heatSimulation[0x40b71f]
[delphi:53489] *** End of error message ***
[delphi:53482] *** Process received signal ***
[delphi:53482] Signal: Segmentation fault (11)
[delphi:53482] Signal code: Address not mapped (1)
[delphi:53482] Failing at address: 0x10
[delphi:53482] [ 0] /lib64/libpthread.so.0(+0xf5d0)[0x7f63ceddf5d0]
[delphi:53482] [ 1] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(+0x17080)[0x7f63ce583080]
[delphi:53482] [ 2] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(+0x171ec)[0x7f63ce5831ec]
[delphi:53482] [ 3] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(SstWriterOpen+0xc6)[0x7f63ce579ba6]
[delphi:53482] [ 4] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios24core6engine9SstWriterC2ERNS0_2IOERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0x167)[0x7f63d0297827]
[delphi:53482] [ 5] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios24core2IO4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0x5f3)[0x7f63cff9e003]
[delphi:53482] [ 6] [delphi:53486] *** Process received signal ***
[delphi:53486] Signal: Segmentation fault (11)
[delphi:53486] Signal code: Address not mapped (1)
[delphi:53486] Failing at address: 0x10
/storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios22IO4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0xe3)[0x7f63d02ff3d3]
[delphi:53482] [ 7] ./heatSimulation[0x40fb36]
[delphi:53482] [ 8] ./heatSimulation[0x40b4ef]
[delphi:53482] [ 9] /lib64/libc.so.6(__libc_start_main+0xf5)[0x7f63cea253d5]
[delphi:53482] [10] ./heatSimulation[0x40b71f]
[delphi:53482] *** End of error message ***
[delphi:53484] *** Process received signal ***
[delphi:53484] Signal: Segmentation fault (11)
[delphi:53484] Signal code: Address not mapped (1)
[delphi:53484] Failing at address: 0x10
[delphi:53486] [ 0] /lib64/libpthread.so.0(+0xf5d0)[0x7fe2f7e8b5d0]
[delphi:53486] [ 1] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(+0x17080)[0x7fe2f762f080]
[delphi:53486] [ 2] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(+0x171ec)[0x7fe2f762f1ec]
[delphi:53486] [ 3] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(SstWriterOpen+0xc6)[0x7fe2f7625ba6]
[delphi:53486] [ 4] [delphi:53487] *** Process received signal ***
[delphi:53487] Signal: Segmentation fault (11)
[delphi:53487] Signal code: Address not mapped (1)
[delphi:53487] Failing at address: 0x10
[delphi:53487] [ 0] [delphi:53496] *** Process received signal ***
[delphi:53496] Signal: Segmentation fault (11)
[delphi:53496] Signal code: Address not mapped (1)
[delphi:53496] Failing at address: 0x10
[delphi:53496] [ 0] [delphi:53498] *** Process received signal ***
[delphi:53498] Signal: Segmentation fault (11)
[delphi:53498] Signal code: Address not mapped (1)
[delphi:53498] Failing at address: 0x10
[delphi:53484] [ 0] /lib64/libpthread.so.0(+0xf5d0)[0x7fa4407215d0]
[delphi:53484] [ 1] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(+0x17080)[0x7fa43fec5080]
[delphi:53484] [ 2] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(+0x171ec)[0x7fa43fec51ec]
[delphi:53484] [ 3] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(SstWriterOpen+0xc6)[0x7fa43febbba6]
[delphi:53484] [ 4] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios24core6engine9SstWriterC2ERNS0_2IOERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0x167)[0x7fa441bd9827]
[delphi:53484] [ 5] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios24core2IO4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0x5f3)[0x7fa4418e0003]
[delphi:53484] [ 6] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios22IO4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0xe3)[0x7fa441c413d3]
[delphi:53484] [ 7] ./heatSimulation[0x40fb36]
[delphi:53484] [ 8] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios24core6engine9SstWriterC2ERNS0_2IOERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0x167)[0x7fe2f9343827]
[delphi:53486] [ 5] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios24core2IO4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0x5f3)[0x7fe2f904a003]
[delphi:53486] [ 6] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios22IO4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0xe3)[0x7fe2f93ab3d3]
[delphi:53486] [ 7] ./heatSimulation[0x40fb36]
[delphi:53486] [ 8] ./heatSimulation[0x40b4ef]
[delphi:53486] [ 9] /lib64/libc.so.6(__libc_start_main+0xf5)[0x7fe2f7ad13d5]
[delphi:53486] [10] ./heatSimulation[0x40b71f]
[delphi:53486] *** End of error message ***
/lib64/libpthread.so.0(+0xf5d0)[0x7efcb0ff15d0]
[delphi:53487] [ 1] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(+0x17080)[0x7efcb0795080]
[delphi:53487] [ 2] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(+0x171ec)[0x7efcb07951ec]
[delphi:53487] [ 3] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(SstWriterOpen+0xc6)[0x7efcb078bba6]
[delphi:53487] [ 4] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios24core6engine9SstWriterC2ERNS0_2IOERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0x167)[0x7efcb24a9827]
[delphi:53487] [ 5] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios24core2IO4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0x5f3)[0x7efcb21b0003]
[delphi:53487] [ 6] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios22IO4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0xe3)[0x7efcb25113d3]
[delphi:53487] [ 7] ./heatSimulation[0x40fb36]
[delphi:53487] [ 8] ./heatSimulation[0x40b4ef]
[delphi:53487] [ 9] /lib64/libc.so.6(__libc_start_main+0xf5)[0x7efcb0c373d5]
[delphi:53487] [10] ./heatSimulation[0x40b71f]
[delphi:53487] *** End of error message ***
/lib64/libpthread.so.0(+0xf5d0)[0x7fa144f785d0]
[delphi:53496] [ 1] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(+0x17080)[0x7fa14471c080]
[delphi:53496] [ 2] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(+0x171ec)[0x7fa14471c1ec]
[delphi:53496] [ 3] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(SstWriterOpen+0xc6)[0x7fa144712ba6]
[delphi:53496] [ 4] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios24core6engine9SstWriterC2ERNS0_2IOERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0x167)[0x7fa146430827]
[delphi:53496] [ 5] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios24core2IO4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0x5f3)[0x7fa146137003]
[delphi:53496] [ 6] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios22IO4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0xe3)[0x7fa1464983d3]
[delphi:53496] [ 7] ./heatSimulation[0x40fb36]
[delphi:53496] [ 8] ./heatSimulation[0x40b4ef]
[delphi:53496] [ 9] /lib64/libc.so.6(__libc_start_main+0xf5)[0x7fa144bbe3d5]
[delphi:53496] [10] [delphi:53498] [ 0] /lib64/libpthread.so.0(+0xf5d0)[0x7f124180d5d0]
[delphi:53498] [ 1] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(+0x17080)[0x7f1240fb1080]
[delphi:53498] [ 2] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(+0x172ff)[0x7f1240fb12ff]
[delphi:53498] [ 3] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(SstReaderOpen+0xaa)[0x7f1240fa49fa]
[delphi:53498] [ 4] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios24core6engine9SstReaderC1ERNS0_2IOERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0x161)[0x7f1242cb2be1]
[delphi:53498] [ 5] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios24core2IO4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0x71e)[0x7f12429cc12e]
[delphi:53498] [ 6] ./heatSimulation[0x40b71f]
[delphi:53496] *** End of error message ***
./heatSimulation[0x40b4ef]
[delphi:53484] [ 9] /lib64/libc.so.6(__libc_start_main+0xf5)[0x7fa4403673d5]
[delphi:53484] [10] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios22IO4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0xe3)[0x7f1242d2d3d3]
[delphi:53498] [ 7] ./heatAnalysis[0x40a827]
[delphi:53498] [ 8] /lib64/libc.so.6(__libc_start_main+0xf5)[0x7f12414533d5]
[delphi:53498] [ 9] ./heatAnalysis[0x40ba5f]
[delphi:53498] *** End of error message ***
./heatSimulation[0x40b71f]
[delphi:53484] *** End of error message ***
[delphi:53499] *** Process received signal ***
[delphi:53499] Signal: Segmentation fault (11)
[delphi:53499] Signal code: Address not mapped (1)
[delphi:53499] Failing at address: 0x10
[delphi:53499] [ 0] /lib64/libpthread.so.0(+0xf5d0)[0x7f1b289265d0]
[delphi:53499] [ 1] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(+0x17080)[0x7f1b280ca080]
[delphi:53499] [ 2] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(+0x172ff)[0x7f1b280ca2ff]
[delphi:53499] [ 3] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(SstReaderOpen+0xaa)[0x7f1b280bd9fa]
[delphi:53499] [ 4] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios24core6engine9SstReaderC1ERNS0_2IOERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0x161)[0x7f1b29dcbbe1]
[delphi:53499] [ 5] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios24core2IO4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0x71e)[0x7f1b29ae512e]
[delphi:53499] [ 6] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios22IO4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0xe3)[0x7f1b29e463d3]
[delphi:53499] [ 7] ./heatAnalysis[0x40a827]
[delphi:53499] [ 8] /lib64/libc.so.6(__libc_start_main+0xf5)[0x7f1b2856c3d5]
[delphi:53499] [ 9] ./heatAnalysis[0x40ba5f]
[delphi:53499] *** End of error message ***
[delphi:53485] *** Process received signal ***
[delphi:53485] Signal: Segmentation fault (11)
[delphi:53485] Signal code: Address not mapped (1)
[delphi:53485] Failing at address: 0x10
[delphi:53485] [ 0] /lib64/libpthread.so.0(+0xf5d0)[0x7fc7d7b995d0]
[delphi:53485] [ 1] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(+0x17080)[0x7fc7d733d080]
[delphi:53485] [ 2] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(+0x171ec)[0x7fc7d733d1ec]
[delphi:53485] [ 3] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/../lib64/libadios2_sst.so.2(SstWriterOpen+0xc6)[0x7fc7d7333ba6]
[delphi:53485] [ 4] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios24core6engine9SstWriterC2ERNS0_2IOERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0x167)[0x7fc7d9051827]
[delphi:53485] [ 5] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios24core2IO4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0x5f3)[0x7fc7d8d58003]
[delphi:53485] [ 6] /storage/users/khuck/src/ADIOS2/install_mpi/lib64/libadios2.so.2(_ZN6adios22IO4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4ModeEP19ompi_communicator_t+0xe3)[0x7fc7d90b93d3]
[delphi:53485] [ 7] ./heatSimulation[0x40fb36]
[delphi:53485] [ 8] ./heatSimulation[0x40b4ef]
[delphi:53485] [ 9] /lib64/libc.so.6(__libc_start_main+0xf5)[0x7fc7d77df3d5]
[delphi:53485] [10] ./heatSimulation[0x40b71f]
[delphi:53485] *** End of error message ***
--------------------------------------------------------------------------
Primary job terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun noticed that process rank 7 with PID 0 on node delphi exited on signal 11 (Segmentation fault).
--------------------------------------------------------------------------
```
And the backtrace from one of the ranks:
```
[khuck@delphi cpp]$ gdb ./heatSimulation core.56907
ImportError: No module named site
[khuck@delphi cpp]$ module unload python
[khuck@delphi cpp]$ gdb ./heatSimulation core.56907
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-114.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /storage/users/khuck/src/adiosvm/Tutorial/heat2d/cpp/heatSimulation...done.
[New LWP 56907]
[New LWP 56929]
[New LWP 57462]
[New LWP 56941]
[New LWP 57851]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
btCore was generated by `./heatSimulation sim.bp 4 4 64 64 100 10'.
Program terminated with signal 11, Segmentation fault.
#0 0x00007fef071c3080 in fi_ep_bind (flags=0, bfid=0x157b120, ep=0x0) at /usr/include/rdma/fi_endpoint.h:168
168 return ep->fid.ops->bind(&ep->fid, bfid, flags);
Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.6-13.el7.x86_64 elfutils-libelf-0.172-2.el7.x86_64 elfutils-libs-0.172-2.el7.x86_64 infinipath-psm-3.3-26_g604758e_open.2.el7.x86_64 keyutils-libs-1.5.8-3.el7.x86_64 krb5-libs-1.15.1-37.el7_6.x86_64 libattr-2.4.46-13.el7.x86_64 libcap-2.22-9.el7.x86_64 libcom_err-1.42.9-13.el7.x86_64 libfabric-1.6.1-2.el7.x86_64 libffi-3.0.13-18.el7.x86_64 libibverbs-17.2-3.el7.x86_64 libnl3-3.2.28-4.el7.x86_64 libpciaccess-0.14-1.el7.x86_64 libpsm2-10.3.58-1.el7.x86_64 librdmacm-17.2-3.el7.x86_64 libselinux-2.5-14.1.el7.x86_64 libsodium13-1.0.5-1.el7.x86_64 libuuid-2.23.2-59.el7_6.1.x86_64 numactl-libs-2.0.9-7.el7.x86_64 openpgm-5.2.122-2.el7.x86_64 pcre-8.32-17.el7.x86_64 systemd-libs-219-62.el7.x86_64 zeromq-4.1.4-5.el7.x86_64 zlib-1.2.7-18.el7.x86_64
(gdb) bt
#0 0x00007fef071c3080 in fi_ep_bind (flags=0, bfid=0x157b120, ep=0x0) at /usr/include/rdma/fi_endpoint.h:168
#1 init_fabric (fabric=0x1578c40, Params=<optimized out>)
at /home/users/khuck/src/ADIOS2/source/adios2/toolkit/sst/dp/rdma_dp.c:198
#2 0x00007fef071c31ec in RdmaInitWriter (Svcs=0x7fef073cbac0 <Svcs>, CP_Stream=0x155c420, Params=0x155c2f8)
at /home/users/khuck/src/ADIOS2/source/adios2/toolkit/sst/dp/rdma_dp.c:558
#3 0x00007fef071b9ba6 in SstWriterOpen (Name=Name@entry=0x155c3e0 "sim.bp", Params=Params@entry=0x155c2f8,
comm=comm@entry=0x155a710) at /home/users/khuck/src/ADIOS2/source/adios2/toolkit/sst/cp/cp_writer.c:1124
#4 0x00007fef08ed7827 in adios2::core::engine::SstWriter::SstWriter(adios2::core::IO&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, adios2::Mode, ompi_communicator_t*) ()
at /home/users/khuck/src/ADIOS2/source/adios2/engine/sst/SstWriter.cpp:35
#5 0x00007fef08bde003 in construct<adios2::core::engine::SstWriter, adios2::core::IO&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, adios2::Mode const&, ompi_communicator_t*&>
(this=<optimized out>, __p=0x155c230) at /storage/packages/gcc/8.1/include/c++/8.1.0/new:169
#6 construct<adios2::core::engine::SstWriter, adios2::core::IO&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, adios2::Mode const&, ompi_communicator_t*&> (__a=...,
__p=0x155c230) at /storage/packages/gcc/8.1/include/c++/8.1.0/bits/alloc_traits.h:475
#7 _Sp_counted_ptr_inplace<adios2::core::IO&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, adios2::Mode const&, ompi_communicator_t*&> (__a=..., this=0x155c220)
at /storage/packages/gcc/8.1/include/c++/8.1.0/bits/shared_ptr_base.h:549
#8 __shared_count<adios2::core::engine::SstWriter, std::allocator<adios2::core::engine::SstWriter>, adios2::core::IO&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, adios2::Mode const&, ompi_communicator_t*&> (__a=..., this=<optimized out>)
at /storage/packages/gcc/8.1/include/c++/8.1.0/bits/shared_ptr_base.h:662
#9 __shared_ptr<std::allocator<adios2::core::engine::SstWriter>, adios2::core::IO&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, adios2::Mode const&, ompi_communicator_t*&> (
__a=..., __tag=..., this=<optimized out>)
at /storage/packages/gcc/8.1/include/c++/8.1.0/bits/shared_ptr_base.h:1328
#10 shared_ptr<std::allocator<adios2::core::engine::SstWriter>, adios2::core::IO&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, adios2::Mode const&, ompi_communicator_t*&> (
__a=..., __tag=..., this=<optimized out>)
at /storage/packages/gcc/8.1/include/c++/8.1.0/bits/shared_ptr.h:360
---Type <return> to continue, or q <return> to quit---
#11 allocate_shared<adios2::core::engine::SstWriter, std::allocator<adios2::core::engine::SstWriter>, adios2::core::IO&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, adios2::Mode const&, ompi_communicator_t*&> (__a=...)
at /storage/packages/gcc/8.1/include/c++/8.1.0/bits/shared_ptr.h:707
#12 make_shared<adios2::core::engine::SstWriter, adios2::core::IO&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, adios2::Mode const&, ompi_communicator_t*&> ()
at /storage/packages/gcc/8.1/include/c++/8.1.0/bits/shared_ptr.h:723
#13 adios2::core::IO::Open(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, adios2::Mode, ompi_communicator_t*) () at /home/users/khuck/src/ADIOS2/source/adios2/core/IO.cpp:567
#14 0x00007fef08f3f3d3 in adios2::IO::Open(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, adios2::Mode, ompi_communicator_t*) ()
at /home/users/khuck/src/ADIOS2/bindings/CXX11/adios2/cxx11/IO.cpp:112
#15 0x000000000040fb36 in IO::IO(Settings const&, ompi_communicator_t*) () at simulation/IO_adios2.cpp:79
#16 0x000000000040b4ef in main () at simulation/heatSimulation.cpp:78
#17 0x00007fef076653d5 in __libc_start_main (main=0x40b320 <main>, argc=8, argv=0x7ffe942a12e8,
init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffe942a12d8)
at ../csu/libc-start.c:266
#18 0x000000000040b71f in _start () at simulation/IO_adios2.cpp:118
```Eisenhauer, GregEisenhauer, Greghttps://code.ornl.gov/pnb/ADIOS2/-/issues/913SST has a 2020-03-03T18:07:17ZPodhorszki, NorbertSST has a *Created by: pnorbert*
SST has the same scalability issue that InsituMPI had in
Fix scalability bottleneck in InSituMPIReader PR #863
"Currently, the reader receives and deserializes messages from each writer one by one. This des...*Created by: pnorbert*
SST has the same scalability issue that InsituMPI had in
Fix scalability bottleneck in InSituMPIReader PR #863
"Currently, the reader receives and deserializes messages from each writer one by one. This design is limiting the scalability in terms of the number of writers per reader. Need to change the behavior to a bulk receive followed by a bulk deserialization."
ffs_marshal.c:
```
static SstStatusValue WaitForReadRequests(SstStream Stream)
{
struct FFSReaderMarshalBase *Info = Stream->ReaderMarshalData;
for (int i = 0; i < Stream->WriterCohortSize; i++)
{
if (Info->WriterInfo[i].Status == Requested)
{
SstStatusValue Result =
SstWaitForCompletion(Stream, Info->WriterInfo[i].ReadHandle);
if (Result == SstSuccess)
{
Info->WriterInfo[i].Status = Full;
DecodeAndPrepareData(Stream, i);
}
else
{
CP_verbose(Stream, "Wait for remote read completion failed, "
"returning failure\n");
return Result;
}
}
}
CP_verbose(Stream, "All remote memory reads completed\n");
return SstSuccess;
}
```
Similarly, this needs to be checked for BP marshalling.Eisenhauer, GregEisenhauer, Greghttps://code.ornl.gov/pnb/ADIOS2/-/issues/1111SST: reading when there are zero size blocks2019-06-13T16:40:51ZPodhorszki, NorbertSST: reading when there are zero size blocks*Created by: pnorbert*
It seems SST reading does not bring the correct data when some processes write zero size blocks.
The sick example was seen in EPOCH, where some variables' data is only written from rank 0 but actually all proces...*Created by: pnorbert*
It seems SST reading does not bring the correct data when some processes write zero size blocks.
The sick example was seen in EPOCH, where some variables' data is only written from rank 0 but actually all processes define and put() the variable. All non-zero rank writes a count=0 size array.
This causes no problem at reading with BPFile. But when reading with SST, the data is invalid.
There is no test or example at the moment to re-create this problem.
In general, zero-sized blocks are allowed as it regularly happens in particle codes (picongpu, xgc, etc).
We need tests like this. Eisenhauer, GregEisenhauer, Greghttps://code.ornl.gov/pnb/ADIOS2/-/issues/2000Starting with ADIOS22020-03-06T19:02:19ZPodhorszki, NorbertStarting with ADIOS2*Created by: aruhela*
What is the recommended way to start with ADIOS2?
Is there available any Video tutorial to get an overview and basic usage of it ?*Created by: aruhela*
What is the recommended way to start with ADIOS2?
Is there available any Video tutorial to get an overview and basic usage of it ?https://code.ornl.gov/pnb/ADIOS2/-/issues/1565Static build for ADIOS2 on Summit: Enet not found error during installion2019-07-19T18:35:24ZPodhorszki, NorbertStatic build for ADIOS2 on Summit: Enet not found error during installion*Created by: swatisgupta*
@eisenhauer
Hi,
The static build for ADIOS2 failed to install on Summit after compilation - It fails to find Enet. I used GNU compilers version 6.4.0. The output from make install can be found below.
...*Created by: swatisgupta*
@eisenhauer
Hi,
The static build for ADIOS2 failed to install on Summit after compilation - It fails to find Enet. I used GNU compilers version 6.4.0. The output from make install can be found below.
make install
[ 4%] Built target adios2sys_objects
[ 6%] Built target atl
[ 11%] Built target dill
[ 21%] Built target ffs
[ 22%] Built target docs
[ 26%] Built target enet
[ 37%] Built target EVPath
[ 38%] Built target cmudp
[ 39%] Built target cmselect
[ 40%] Built target cmsockets
[ 41%] Built target cmenet
[ 42%] Built target cmmulticast
[ 43%] Built target cmib
[ 44%] Built target taustubs
[ 48%] Built target sst
[ 81%] Built target adios2
[ 82%] Built target sst_conn_tool
[ 84%] Built target adios_reorganize
[ 85%] Built target bpls
[ 88%] Built target adios_iotest
[100%] Built target adios2_f
Install the project...
-- Install configuration: "Release"
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/common/ADIOSConfig.h
Pre-installation cleanup of CMake files
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/CMakeFindDependencyMacro.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/FindBZip2.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/FindBlosc.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/FindCrayDRC.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/FindHDF5.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/FindLIBFABRIC.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/FindMGARD.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/FindMPI.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/FindPkgConfig.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/FindPythonFull.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/FindPythonModule.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/FindSZ.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/FindZFP.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/FindZeroMQ.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/upstream
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/upstream/FindBZip2.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/upstream/FindHDF5.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/upstream/FindMPI.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/upstream/FindMPI
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/upstream/FindMPI/fortranparam_mpi.f90.in
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/upstream/FindMPI/libver_mpi.c
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/upstream/FindMPI/libver_mpi.f90.in
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/upstream/FindMPI/mpiver.f90.in
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/upstream/FindMPI/test_mpi.c
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/upstream/FindMPI/test_mpi.cxx
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/upstream/FindMPI/test_mpi.f90.in
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/upstream/FindMPI/test_mpi_c.cxx
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/upstream/FindPkgConfig.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/upstream/GoogleTest.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/upstream/CMakeFindDependencyMacro.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/libadios2_atl.a
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/atl-config.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/atl-config-version.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/atl-targets.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/atl-targets-release.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/libadios2_dill.a
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/dill-config-common.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/dill-config-version.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/dill-config.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/dill-targets.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/dill-targets-release.cmake
-- Up-to-date: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/FindBinUtils.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/FindLibFFI.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/libadios2_ffs.a
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/ffs-config.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/ffs-config-version.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/ffs-targets.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/ffs-targets-release.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/libadios2_enet.a
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/enet-config.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/enet-config-version.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/enet-targets.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/enet-targets-release.cmake
-- Up-to-date: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/FindCERCSProject.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/FindIBVERBS.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/FindLIBFABRIC.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/FindNNTI.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/FindNVML.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/FindPkgConfig.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/upstream
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/upstream/FindPkgConfig.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/libadios2_evpath.a
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/libadios2_cmselect.so
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/libadios2_cmsockets.so
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/libadios2_cmudp.so
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/libadios2_cmmulticast.so
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/libadios2_cmenet.so
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/libadios2_cmib.so
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/EVPathTargets.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/EVPathTargets-release.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/EVPathConfigCommon.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/EVPathConfigVersion.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/EVPathConfig.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/common/ADIOSMacros.h
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/common/ADIOSTypes.h
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/common/ADIOSTypes.inl
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/libadios2.a
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/libtaustubs.a
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/toolkit/sst/SSTConfig.h
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/libadios2_sst.a
-- Installing: /ccs/home/ssinghal/ADIOS2-install/bin/sst_conn_tool
-- Installing: /ccs/home/ssinghal/ADIOS2-install/bin/bpls
-- Installing: /ccs/home/ssinghal/ADIOS2-install/bin/adios_reorganize
-- Installing: /ccs/home/ssinghal/ADIOS2-install/bin/adios_iotest
-- Installing: /ccs/home/ssinghal/ADIOS2-install/share/iotest-config
-- Installing: /ccs/home/ssinghal/ADIOS2-install/share/iotest-config/coupling2.xml
-- Installing: /ccs/home/ssinghal/ADIOS2-install/share/iotest-config/coupling2_blocking_01_simple.txt
-- Installing: /ccs/home/ssinghal/ADIOS2-install/share/iotest-config/coupling2_blocking_02_dump_result.txt
-- Installing: /ccs/home/ssinghal/ADIOS2-install/share/iotest-config/pipe2.xml
-- Installing: /ccs/home/ssinghal/ADIOS2-install/share/iotest-config/pipe2_01_simple.txt
-- Installing: /ccs/home/ssinghal/ADIOS2-install/share/iotest-config/pipe2_02_link.txt
-- Installing: /ccs/home/ssinghal/ADIOS2-install/share/iotest-config/pipe2_03_conditional.txt
-- Installing: /ccs/home/ssinghal/ADIOS2-install/share/iotest-config/pipe2_04_nonblocking.txt
-- Installing: /ccs/home/ssinghal/ADIOS2-install/share/iotest-config/pipe2_05_add_variables.txt
-- Installing: /ccs/home/ssinghal/ADIOS2-install/share/iotest-config/wf_2_to_1.txt
-- Installing: /ccs/home/ssinghal/ADIOS2-install/share/iotest-config/wf_2_to_1.xml
-- Installing: /ccs/home/ssinghal/ADIOS2-install/share/iotest-config/pipe2_06_busy.txt
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/libadios2_f.a
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/fortran
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/fortran/adios2_engine_begin_step_mod.mod
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/fortran/adios2_parameters_mod.mod
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/fortran/adios2_functions_allocate_mod.mod
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/fortran/adios2_io_open_mod.mod
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/fortran/adios2_io_define_attribute_mod.mod
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/fortran/adios2_operator_mod.mod
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/fortran/adios2_functions_mod.mod
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/fortran/adios2_attribute_mod.mod
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/fortran/adios2_adios_init_mod.mod
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/fortran/adios2_variable_mod.mod
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/fortran/adios2_variable_min_mod.mod
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/fortran/adios2_io_define_variable_mod.mod
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/fortran/adios2_attribute_data_mod.mod
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/fortran/adios2_engine_put_mod.mod
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/fortran/adios2_variable_max_mod.mod
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/fortran/adios2_engine_get_mod.mod
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/fortran/adios2_adios_mod.mod
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/fortran/adios2_io_mod.mod
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/fortran/adios2_engine_mod.mod
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/fortran/adios2.mod
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2_c.h
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/c/adios2_c_types.h
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/c/adios2_c_adios.h
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/c/adios2_c_io.h
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/c/adios2_c_variable.h
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/c/adios2_c_attribute.h
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/c/adios2_c_engine.h
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/c/adios2_c_operator.h
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2.h
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/cxx11/fstream/ADIOS2fstream.h
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/cxx11/ADIOS.h
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/cxx11/ADIOS.inl
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/cxx11/IO.h
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/cxx11/Variable.h
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/cxx11/Attribute.h
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/cxx11/Engine.h
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/cxx11/Operator.h
-- Installing: /ccs/home/ssinghal/ADIOS2-install/include/adios2/cxx11/Types.h
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/adios2-config-version.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/adios2-config-common.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/adios2-targets.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/adios2-targets-release.cmake
-- Installing: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/adios2-config.cmake
/tmp/tmp.bSbXtooDsS ~/ADIOS2/build
Configuring adios2-config dummy CMake project
-- The C compiler identification is GNU 6.4.0
-- The CXX compiler identification is GNU 6.4.0
-- Check for working C compiler: /autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/spectrum-mpi-10.3.0.0-20190419-4um5hjogm3tepg4xe23hrptlrs2y7ez6/bin/mpicc
-- Check for working C compiler: /autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/spectrum-mpi-10.3.0.0-20190419-4um5hjogm3tepg4xe23hrptlrs2y7ez6/bin/mpicc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/spectrum-mpi-10.3.0.0-20190419-4um5hjogm3tepg4xe23hrptlrs2y7ez6/bin/mpicxx
-- Check for working CXX compiler: /autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/spectrum-mpi-10.3.0.0-20190419-4um5hjogm3tepg4xe23hrptlrs2y7ez6/bin/mpicxx -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- The Fortran compiler identification is GNU 6.4.0
-- Check for working Fortran compiler: /autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/spectrum-mpi-10.3.0.0-20190419-4um5hjogm3tepg4xe23hrptlrs2y7ez6/bin/mpif90
-- Check for working Fortran compiler: /autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/spectrum-mpi-10.3.0.0-20190419-4um5hjogm3tepg4xe23hrptlrs2y7ez6/bin/mpif90 -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether /autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/spectrum-mpi-10.3.0.0-20190419-4um5hjogm3tepg4xe23hrptlrs2y7ez6/bin/mpif90 supports Fortran 90
-- Checking whether /autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/spectrum-mpi-10.3.0.0-20190419-4um5hjogm3tepg4xe23hrptlrs2y7ez6/bin/mpif90 supports Fortran 90 -- yes
-- Found MPI_C: /autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/spectrum-mpi-10.3.0.0-20190419-4um5hjogm3tepg4xe23hrptlrs2y7ez6/bin/mpicc (found version "3.1")
-- Found MPI_CXX: /autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/spectrum-mpi-10.3.0.0-20190419-4um5hjogm3tepg4xe23hrptlrs2y7ez6/bin/mpicxx (found version "3.1")
-- Found MPI_Fortran: /autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/spectrum-mpi-10.3.0.0-20190419-4um5hjogm3tepg4xe23hrptlrs2y7ez6/bin/mpif90 (found version "3.1")
-- Found MPI: TRUE (found version "3.1")
-- Found atl: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/atl-config.cmake (found suitable version "2.2.1", minimum required is "2.2.1")
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.27.1")
-- Checking for module 'libffi'
-- Found libffi, version 3.0.13
-- Found LibFFI: -lffi
-- Found dill: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/dill-config.cmake (found version "2.4.0")
-- Found ffs: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/ffs-config.cmake (found suitable version "1.6.0", minimum required is "1.6.0")
-- Found EVPath: /ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/EVPathConfigCommon.cmake (found version "4.4.0")
CMake Error at /autofs/nccs-svm1_sw/summit/.swci/0-core/opt/spack/20180914/linux-rhel7-ppc64le/gcc-4.8.5/cmake-3.14.2-3zkrrk4chuq6wfhvxpmen2gzdpaoiorw/share/cmake-3.14/Modules/CMakeFindDependencyMacro.cmake:48 (find_package):
**Found package configuration file:
/ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/thirdparty/EVPathConfig.cmake
but it set EVPath_FOUND to FALSE so package "EVPath" is considered to be
NOT FOUND. Reason given by package:
The following imported targets are referenced, but are missing: enet::enet**
Call Stack (most recent call first):
/ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/adios2-config-common.cmake:57 (find_dependency)
/ccs/home/ssinghal/ADIOS2-install/lib64/cmake/adios2/adios2-config.cmake:29 (include)
CMakeLists.txt:5 (find_package)
-- Configuring incomplete, errors occurred!
See also "/tmp/tmp.bSbXtooDsS/CMakeFiles/CMakeOutput.log".
FailedEisenhauer, GregEisenhauer, Greghttps://code.ornl.gov/pnb/ADIOS2/-/issues/1221static build on Mac: install of adios2-config fails2019-06-17T22:30:11ZPodhorszki, Norbertstatic build on Mac: install of adios2-config fails*Created by: pnorbert*
pnb13:build-parallel-static pnb$ cmake -DCMAKE_INSTALL_PREFIX=/opt/adios2-static -DADIOS2_USE_MPI=ON -DBUILD_SHARED_LIBS=OFF -DCMAKE_POSITION_INDEPENDENT_CODE:BOOL=true ..
```
ADIOS2 build configuration:
AD...*Created by: pnorbert*
pnb13:build-parallel-static pnb$ cmake -DCMAKE_INSTALL_PREFIX=/opt/adios2-static -DADIOS2_USE_MPI=ON -DBUILD_SHARED_LIBS=OFF -DCMAKE_POSITION_INDEPENDENT_CODE:BOOL=true ..
```
ADIOS2 build configuration:
ADIOS Version: 2.3.1
C++ Compiler : AppleClang 9.0.0.9000038
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
Fortran Compiler : GNU 8.2.0
/usr/local/bin/gfortran
Installation prefix: /opt/adios2-static
bin: bin
lib: lib
include: include
cmake: lib/cmake/adios2
Features:
Library Type: static (with PIC)
Build Type: Release
Testing: ON
Build Options:
BZip2 : ON
ZFP : OFF
SZ : OFF
MGARD : OFF
MPI : ON
DataMan : ON
SST : ON
ZeroMQ : OFF
HDF5 : ON
Python : OFF
Fortran : ON
SysVShMem: ON
Endian_Reverse: OFF
```
```
$ sudo make install
...
-- Found atl: /opt/adios2-static/lib/cmake/adios2/thirdparty/atl-config.cmake (found suitable version "2.2.1", minimum required is "2.2.1")
-- Found atl: /opt/adios2-static/lib/cmake/adios2/thirdparty/atl-config.cmake (found version "2.2.1")
-- Found dill: /opt/adios2-static/lib/cmake/adios2/thirdparty/dill-config.cmake (found version "2.4.0")
-- Found ffs: /opt/adios2-static/lib/cmake/adios2/thirdparty/ffs-config.cmake (found suitable version "1.6.0", minimum required is "1.6.0")
-- Found dill: /opt/adios2-static/lib/cmake/adios2/thirdparty/dill-config.cmake (found suitable version "2.4.0", minimum required is "2.4.0")
-- Found EVPath: /opt/adios2-static/lib/cmake/adios2/thirdparty/EVPathConfigCommon.cmake (found version "4.4.0")
CMake Error at /opt/adios2-static/lib/cmake/adios2/upstream/CMakeFindDependencyMacro.cmake:47 (find_package):
Found package configuration file:
/opt/adios2-static/lib/cmake/adios2/thirdparty/EVPathConfig.cmake
but it set EVPath_FOUND to FALSE so package "EVPath" is considered to be
NOT FOUND. Reason given by package:
The following imported targets are referenced, but are missing: enet::enet
Call Stack (most recent call first):
/opt/adios2-static/lib/cmake/adios2/adios2-config-common.cmake:57 (find_dependency)
/opt/adios2-static/lib/cmake/adios2/adios2-config.cmake:29 (include)
CMakeLists.txt:5 (find_package)
-- Configuring incomplete, errors occurred!
See also "/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/tmp.bxDz4Tym/CMakeFiles/CMakeOutput.log".
Failed
```2.4.0 Production Releasehttps://code.ornl.gov/pnb/ADIOS2/-/issues/1524summit anaconda build error2019-08-09T01:47:05ZPodhorszki, Norbertsummit anaconda build error*Created by: zeneofa*
Hi,
I am having problems trying to build adios2 on summit, using the anaconda environment. I seems that the adios2.cpython-36m-powerpc64le-linux-gnu.so file is not created all. I am building from the github mas...*Created by: zeneofa*
Hi,
I am having problems trying to build adios2 on summit, using the anaconda environment. I seems that the adios2.cpython-36m-powerpc64le-linux-gnu.so file is not created all. I am building from the github master branch
```
╰─[:)] % git rev-parse HEAD
cbaf8ca92b4550cad603522e6570f3cbf95acdba
```
The modules that I have loaded:
```
Currently Loaded Modules:
1) hsi/5.0.2.p5 3) lsf-tools/2.0 5) DefApps 7) python/3.6.6-anaconda3-5.3.0 9) cmake/3.14.2 11) magma/2.3.0
2) xalt/1.1.3 4) darshan-runtime/3.1.7 6) cuda/10.1.105 8) gcc/6.4.0 10) spectrum-mpi/10.3.0.0-20190419
```
I use the following command to build
```
mkdir adios2-build
cd adios2-build
cmake -DCMAKE_C_COMPILER=${OMPI_CC} -DCMAKE_CXX_COMPILER=${OMPI_CXX} -DCMAKE_INSTALL_PREFIX=~/software/adios2 -DADIOS2_USE_Fortran=OFF -DADIOS2_USE_ZeroMQ=OFF -DBUILD_SHARED_LIBS=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DADIOS2_USE_SST=OFF .. 2>&1 | tee cmake.configure.adios2.log
```
After this I run
```
make -j 1 2>&1 | tee make.adios2.log
```
which then errors with:
```
[ 38%] Building CXX object bindings/Python/CMakeFiles/adios2py.dir/py11glue.cpp.o
[ 39%] Linking CXX shared module ../../lib64/python3.6/site-packages/adios2.cpython-36m-powerpc64le-linux-gnu.so
lto1: internal compiler error: invalid line in the resolution file
0x101d225b lto_resolution_read
/sw/summit/gcc/6.4.0/src/gcc-6.4.0/gcc/lto/lto.c:1912
0x101d225b lto_file_read
/sw/summit/gcc/6.4.0/src/gcc-6.4.0/gcc/lto/lto.c:2092
0x101d225b read_cgraph_and_symbols
/sw/summit/gcc/6.4.0/src/gcc-6.4.0/gcc/lto/lto.c:2806
0x101d225b lto_main()
/sw/summit/gcc/6.4.0/src/gcc-6.4.0/gcc/lto/lto.c:3310
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <http://gcc.gnu.org/bugs.html> for instructions.
lto-wrapper: fatal error: /sw/summit/gcc/6.4.0/bin/g++ returned 1 exit status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
/usr/bin/sha1sum: ../../lib64/python3.6/site-packages/adios2.cpython-36m-powerpc64le-linux-gnu.so: No such file or directory
collect2: error: ld returned 1 exit status
make[2]: *** [lib64/python3.6/site-packages/adios2.cpython-36m-powerpc64le-linux-gnu.so] Error 1
make[1]: *** [bindings/Python/CMakeFiles/adios2py.dir/all] Error 2
make: *** [all] Error 2
```
I have loaded the output logs on gist here: https://gist.github.com/zeneofa/9c6d8ccb102a8c21f45032dfce29846a
This seems to be anaconda specific, as when I use the standard python module I do not get the lto error.
```
module unload python/3.6.6-anaconda3-5.3.0
module load python/3.7.0
```
This is independent of python version, using the anaconda 3.7 version also fails.
Any help would be much appreciated. Thanks in advance.
Phttps://code.ornl.gov/pnb/ADIOS2/-/issues/748Support for string arrays2018-07-26T03:01:24ZPodhorszki, NorbertSupport for string arrays*Created by: isosc*
This is a followup on a discussion with @williamfgc and @pnorbert about adding support for string arrays (vector of strings, numpy array). The primary need is to extend the C++ API to support put and get of vectors o...*Created by: isosc*
This is a followup on a discussion with @williamfgc and @pnorbert about adding support for string arrays (vector of strings, numpy array). The primary need is to extend the C++ API to support put and get of vectors of strings. Once that exists, extending this functionality into Python should be straightforward.
One design issue is how to serialize and deserialize this data structure to BP. We discussed a tentative strategy which involved storing a list of string sizes followed by the collapsed string data. https://code.ornl.gov/pnb/ADIOS2/-/issues/1274Tools Interface in ADIOS22019-04-24T18:32:22ZPodhorszki, NorbertTools Interface in ADIOS2*Created by: rtschueter*
In the last commits work on a TAU interface for ADIOS has started. Currently, the interface seems tailored to TAU. However, a generic tools interface would be of great value for performance tools in general. Ar...*Created by: rtschueter*
In the last commits work on a TAU interface for ADIOS has started. Currently, the interface seems tailored to TAU. However, a generic tools interface would be of great value for performance tools in general. Are there any plans to provide a tools interface similar to the one of ADIOS1? On the one hand, there might be potential for cooperation between developers from different performance tools. On the other hand, developers of ADIOS and performance tools can discuss about supporting each other, e.g., what performance data should be provided by the tools.https://code.ornl.gov/pnb/ADIOS2/-/issues/1966Trouble reading files in python2020-02-14T18:52:53ZPodhorszki, NorbertTrouble reading files in python*Created by: rkube*
I'm trying to read multiple bp files in python. I have filex xgc.3d.000??.c.bp where ??=0..92.
This is the content of the files:
(ml) [rkube@traverse ml_data_case_1_92ts_iteration]$ bpls xgc.3d.00092.c.bp
doub...*Created by: rkube*
I'm trying to read multiple bp files in python. I have filex xgc.3d.000??.c.bp where ??=0..92.
This is the content of the files:
(ml) [rkube@traverse ml_data_case_1_92ts_iteration]$ bpls xgc.3d.00092.c.bp
double a_par {19316, 8}
double apar_del {19316, 8}
double apar_old {19316, 8}
double apar_res {19316, 8}
double apar_try {19316, 8}
double dadt {19316, 8}
double dn_del {19316, 8}
double dpot {19316, 8}
double eden {19316, 8}
double iden {19316, 8}
int32_t iphi scalar
int32_t nnode scalar
int32_t nphi scalar
double pot0 [1]*{19316}
double pot_del {19316, 8}
double pot_mid {19316, 8}
double pot_old {19316, 8}
double pot_res {19316, 8}
double pot_try {19316, 8}
double potm0 [1]*{19316}
double u_e {19316, 8}
double u_i {19316, 8}
When I'm trying to access the file using the example code from the docs I don't see any contents:
```python
print(fname)
with adios2.open(fname, "r") as fh:
for fstep in fh:
step_vars = fstep.available_variables()
for name, info in step_vars.items():
print("variable_name: " + name)
for key, value in info.items():
print("\t" + key + ": " + value)
print("\n")
```
The command above doesn't print anythin
When I try to read data directly from fh it works fine:
```python
data = rh.read("a_par")
```
When I try reading the fields from fh in a loop I get an error:
```
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-61-094e1f12f241> in <module>
2 fname = join(datapath, f"xgc.3d.{nn_idx:05d}.c.bp")
3 with adios2.open(fname, "r") as fh:
----> 4 apar = fh.read("a_par")
5 dpot = fh.read("dpot")
6 fh.close()
ValueError: ERROR: offset 0 from steps start 41 in variable a_par is beyond the largest available step = 41, check Variable SetStepSelection argument stepsCount (random access), or number of BeginStep calls (streaming), in call to Get
```
How can I consistently read the data from my files?https://code.ornl.gov/pnb/ADIOS2/-/issues/1316Two writers in one I/O2019-03-22T20:23:34ZPodhorszki, NorbertTwo writers in one I/O*Created by: germasch*
The last commit of PR #1312 adds the following test:
```cxx
auto writer = this->m_Io.Open(filename, adios2::Mode::Write);
auto writer2 = this->m_Io.Open("multi_put2writers2.bp", adios2::Mode::Write);
...*Created by: germasch*
The last commit of PR #1312 adds the following test:
```cxx
auto writer = this->m_Io.Open(filename, adios2::Mode::Write);
auto writer2 = this->m_Io.Open("multi_put2writers2.bp", adios2::Mode::Write);
auto var = this->m_Io.template DefineVariable<T>("var", this->m_Shape);
MyData<T> myData(this->m_Selections);
for (int b = 0; b < myData.NBlocks(); ++b)
{
this->PopulateBlock(myData, b);
var.SetSelection(myData.Selection(b));
writer.Put(var, &myData[b][0], TypeParam::PutMode);
writer2.Put(var, &myData[b][0], TypeParam::PutMode);
}
writer2.Close();
writer.Close();
```
The test is commented out in the PR because it produces invalid output.
Now, I know this use case is somewhat far-fetched. However:
1. Multiple engines per I/O are supported by the API, so I suppose if you let the app use adios2 this way, it should work correctly.
2. The test writes the same data into two files, which is not a realistic use case. However, in one of my apps, I calculate moments of the particle distribution. I write them into one output file as a snapshot just for the current time step, but I also calculate an average of the past n timesteps and write those smoothed moments into a separate file, so this pattern actually occurs in real life.
https://code.ornl.gov/pnb/ADIOS2/-/issues/2015Update Keys of Variables/Attributes2020-03-03T17:33:07ZPodhorszki, NorbertUpdate Keys of Variables/Attributes*Created by: ax3l*
**Why is this feature important?**
We develop the [openPMD-standard](https://github.com/openPMD/openPMD-standard) as a meta-data schema for particle-mesh codes.
As the standard evolves, we can [update the meta-dat...*Created by: ax3l*
**Why is this feature important?**
We develop the [openPMD-standard](https://github.com/openPMD/openPMD-standard) as a meta-data schema for particle-mesh codes.
As the standard evolves, we can [update the meta-data](https://github.com/openPMD/openPMD-updater/) of existing files to a newer version by:
- renaming paths of variables and attributes
- removing (undefining/hiding) variables/attributes
- adding new attributes
This problem will commonly occur when users open "old" openPMD files in an openPMD reader that supports only a newer version of the standard. A quick, in-place meta-data conversion simplifies reader implementations and provides a convenient user-experience.
**What is the potential impact of this feature in the community?**
This allows faster development of data readers of meta-data schemes (only need to implement the newest version of a schema) and a better user experience with forward-updateable meta-data.
**Is your feature request related to a problem? Please describe.**
Related to openPMD:
- https://www.openpmd.org
- https://github.com/openPMD/openPMD-standard
**Describe the solution you'd like and potential required effort**
I would like to update the "index table" of variables and attributes, e.g. with an update to the file. It's sufficient to "hide" old meta-data, e.g. as revision of a file.
There is no need to remove actual data, maybe a repack command (`h5repack`-like or `adios_reorganize`) that actually copies data could remove old revisions and un-indexed data when its actually needed.
**Describe alternatives you've considered and potential required effort**
An alternative is to copy all data to a new file (series) on a meta-data update.
**Additional context**
cc @pnorbert @chuckatkins @franzpoeschel https://code.ornl.gov/pnb/ADIOS2/-/issues/1290Variable::Span<T>2019-03-19T13:07:09ZPodhorszki, NorbertVariable::Span<T>*Created by: germasch*
Can you provide some context on how apps are expected to use `Span<T>` from #1289? I gather there have been some requests for this, but I'd be interested to learn how others use this.
It looks like a useful fea...*Created by: germasch*
Can you provide some context on how apps are expected to use `Span<T>` from #1289? I gather there have been some requests for this, but I'd be interested to learn how others use this.
It looks like a useful feature to me, which I might want to use in my projects.
The way it looks to me from the test is:
```cxx
var = io.DefineVariable<double>(....)
writer = io.Open(...)
auto span = writer.Put<T>(var)
// fill span with data
// How do I say that I'm done filling the data and it's ready to be written? Isn't that what Put is for?
```
The other part I'm wondering about is, why std::span (well, that's C++20, but you know what I mean)? On the app side, that works for simple 1-d arrays, I guess. But at lot of apps do more complicated things.
In my case, as an example, I calculate moments from my particle data as diagnostics. So I create a 2-d or 3-d array, calculate my output data in it, then write it, then destroy it. I think the point of this is to avoid me allocating data, then handing it to ADIOS2, which will at some point copy it into its internal buffer, then I free my data. It'd be nicer if I could create my 2-d/3-d data structure directly using adios2's buffer, avoiding the need to have two temporary buffers and a copy. But std::span doesn't really help me very much, or what's the idea?
https://code.ornl.gov/pnb/ADIOS2/-/issues/794Zeros in multiple time steps when no Close is called and Flush is (BPFIle)2019-09-25T17:14:40ZPodhorszki, NorbertZeros in multiple time steps when no Close is called and Flush is (BPFIle)*Created by: stevenwalton*
Working with the Ascent program I am unable to call `adiosWriter.Close()` and a few issues have been brought up because of this case. I have found that when writing to BPFiles that the first time step correctl...*Created by: stevenwalton*
Working with the Ascent program I am unable to call `adiosWriter.Close()` and a few issues have been brought up because of this case. I have found that when writing to BPFiles that the first time step correctly writes, but each subsequent `BeginStep()/EndStep()` pair is called that the data does not get written.
This currently does not affect SST or DataMan and #757 fixed the issues for HDF5.
When writing toy examples I can not reproduce the problem and I do not get it when I use small vectors within Ascent. It appears that vector sizes need to be quite large to reproduce this error.
If `adiosWriter.Close()` is called, this is not an issue.