Skip to content

GitLab

  • Menu
Projects Groups Snippets
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in
  • A ADIOS2
  • Project information
    • Project information
    • Activity
    • Labels
    • Planning hierarchy
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 97
    • Issues 97
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
  • Merge requests 9
    • Merge requests 9
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Packages & Registries
    • Packages & Registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Issue
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • Podhorszki, Norbert
  • ADIOS2
  • Issues
  • #2015

Closed
Open
Created Mar 03, 2020 by Podhorszki, Norbert@pnbMaintainer

Update Keys of Variables/Attributes

Created by: ax3l

Why is this feature important?

We develop the openPMD-standard as a meta-data schema for particle-mesh codes. As the standard evolves, we can update the meta-data of existing files to a newer version by:

  • renaming paths of variables and attributes
  • removing (undefining/hiding) variables/attributes
  • adding new attributes

This problem will commonly occur when users open "old" openPMD files in an openPMD reader that supports only a newer version of the standard. A quick, in-place meta-data conversion simplifies reader implementations and provides a convenient user-experience.

What is the potential impact of this feature in the community?

This allows faster development of data readers of meta-data schemes (only need to implement the newest version of a schema) and a better user experience with forward-updateable meta-data.

Is your feature request related to a problem? Please describe.

Related to openPMD:

  • https://www.openpmd.org
  • https://github.com/openPMD/openPMD-standard

Describe the solution you'd like and potential required effort

I would like to update the "index table" of variables and attributes, e.g. with an update to the file. It's sufficient to "hide" old meta-data, e.g. as revision of a file.

There is no need to remove actual data, maybe a repack command (h5repack-like or adios_reorganize) that actually copies data could remove old revisions and un-indexed data when its actually needed.

Describe alternatives you've considered and potential required effort

An alternative is to copy all data to a new file (series) on a meta-data update.

Additional context

cc @pnorbert @chuckatkins @franzpoeschel

Assignee
Assign to
Time tracking