Dynamic variable dimension

Created by: sungsooha

In adios1, the dimension of a variable could be defined with another variable such that

import adios_mpi as ad
ad.init_noxml()
g = ad.declare_group("myGroup", "", ad.FLAG.YES)
ad.define_var(g, "y_dim", "", ad.DATATYPE.unsigned_integer, "", "", "")
ad.define_var(g, "data" , "", ad.DATATYPE.unsigned_integer, "y_dim,6", "y_dim,6", "0,0")

Q1. I wonder if there is a way to achieve this in adios2?

Let's say there are five steps of writing for both y_dim and data. So, y_dim = [22, 100, 400, 300, 12]. In adios2, I read the data as follows:

from mpi4py import MPI
import adios2
import numpy as np

comm = MPI.COMM_WORLD
adios = adios2.ADIOS(comm)
io = adios.DeclareIO("bpReader")
stream = io.Open("fn.bp", adios2.Mode.Read)

varInfo = io.AvailableVariables()
steps = int(varInfo["y_dim"]["AvailableStepsCount"])

var_y_dim = io.InquireVariable("y_dim")
var_data = io.InquireVariable("data")

for step in range(steps):
     stream.BeginStep()
     y_dim = np.zeros(1, dtype=np.uint32)
     stream.Get(var_y_dim, y_dim)
     stream.PerformGets()

     data = np.zeros((y_dim[0], 6), dtype=np.uint32)
     stream.Get(var_data, data)
     stream.PerformGets()
     stream.EndStep()

In the above code, I could correctly read data only for the first step. From the second to fourth step, I am only able to get the first 22 rows. And for the fifth step, I got an error saying ValueError: ERROR: selection Start Dims(2):[0,0] and Count Dims(2):[22,6] (requested) is out of bounds of (available) Shape Dims(2):[12, 6], when reading global array variable data, in call to Get.

Q2. Is there a way to read data of a variable whose dimension is dynamically changing while it is written?

Any suggestion or comments would be a lot appreciated.

Thanks, Sungsoo