Dynamic variable dimension
Created by: sungsooha
In adios1, the dimension of a variable could be defined with another variable such that
import adios_mpi as ad
ad.init_noxml()
g = ad.declare_group("myGroup", "", ad.FLAG.YES)
ad.define_var(g, "y_dim", "", ad.DATATYPE.unsigned_integer, "", "", "")
ad.define_var(g, "data" , "", ad.DATATYPE.unsigned_integer, "y_dim,6", "y_dim,6", "0,0")
Q1. I wonder if there is a way to achieve this in adios2?
Let's say there are five steps of writing for both y_dim
and data
. So, y_dim = [22, 100, 400, 300, 12]
.
In adios2, I read the data as follows:
from mpi4py import MPI
import adios2
import numpy as np
comm = MPI.COMM_WORLD
adios = adios2.ADIOS(comm)
io = adios.DeclareIO("bpReader")
stream = io.Open("fn.bp", adios2.Mode.Read)
varInfo = io.AvailableVariables()
steps = int(varInfo["y_dim"]["AvailableStepsCount"])
var_y_dim = io.InquireVariable("y_dim")
var_data = io.InquireVariable("data")
for step in range(steps):
stream.BeginStep()
y_dim = np.zeros(1, dtype=np.uint32)
stream.Get(var_y_dim, y_dim)
stream.PerformGets()
data = np.zeros((y_dim[0], 6), dtype=np.uint32)
stream.Get(var_data, data)
stream.PerformGets()
stream.EndStep()
In the above code, I could correctly read data
only for the first step. From the second to fourth step, I am only able to get the first 22 rows. And for the fifth step, I got an error saying ValueError: ERROR: selection Start Dims(2):[0,0] and Count Dims(2):[22,6] (requested) is out of bounds of (available) Shape Dims(2):[12, 6], when reading global array variable data, in call to Get
.
Q2. Is there a way to read data of a variable whose dimension is dynamically changing while it is written?
Any suggestion or comments would be a lot appreciated.
Thanks, Sungsoo