Update Keys of Variables/Attributes

Created by: ax3l

Why is this feature important?

We develop the openPMD-standard as a meta-data schema for particle-mesh codes. As the standard evolves, we can update the meta-data of existing files to a newer version by:

  • renaming paths of variables and attributes
  • removing (undefining/hiding) variables/attributes
  • adding new attributes

This problem will commonly occur when users open "old" openPMD files in an openPMD reader that supports only a newer version of the standard. A quick, in-place meta-data conversion simplifies reader implementations and provides a convenient user-experience.

What is the potential impact of this feature in the community?

This allows faster development of data readers of meta-data schemes (only need to implement the newest version of a schema) and a better user experience with forward-updateable meta-data.

Is your feature request related to a problem? Please describe.

Related to openPMD:

Describe the solution you'd like and potential required effort

I would like to update the "index table" of variables and attributes, e.g. with an update to the file. It's sufficient to "hide" old meta-data, e.g. as revision of a file.

There is no need to remove actual data, maybe a repack command (h5repack-like or adios_reorganize) that actually copies data could remove old revisions and un-indexed data when its actually needed.

Describe alternatives you've considered and potential required effort

An alternative is to copy all data to a new file (series) on a meta-data update.

Additional context

cc @pnorbert @chuckatkins @franzpoeschel