Skip to content

GitLab

  • Menu
Projects Groups Snippets
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in
  • A ADIOS2
  • Project information
    • Project information
    • Activity
    • Labels
    • Planning hierarchy
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 97
    • Issues 97
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
  • Merge requests 9
    • Merge requests 9
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Packages & Registries
    • Packages & Registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Issue
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • Podhorszki, Norbert
  • ADIOS2
  • Issues
  • #1966

Closed
Open
Created Feb 13, 2020 by Podhorszki, Norbert@pnbMaintainer

Trouble reading files in python

Created by: rkube

I'm trying to read multiple bp files in python. I have filex xgc.3d.000??.c.bp where ??=0..92.

This is the content of the files: (ml) [rkube@traverse ml_data_case_1_92ts_iteration]$ bpls xgc.3d.00092.c.bp double a_par {19316, 8} double apar_del {19316, 8} double apar_old {19316, 8} double apar_res {19316, 8} double apar_try {19316, 8} double dadt {19316, 8} double dn_del {19316, 8} double dpot {19316, 8} double eden {19316, 8} double iden {19316, 8} int32_t iphi scalar int32_t nnode scalar int32_t nphi scalar double pot0 [1]{19316} double pot_del {19316, 8} double pot_mid {19316, 8} double pot_old {19316, 8} double pot_res {19316, 8} double pot_try {19316, 8} double potm0 [1]{19316} double u_e {19316, 8} double u_i {19316, 8}

When I'm trying to access the file using the example code from the docs I don't see any contents:

print(fname)
with adios2.open(fname, "r") as fh:            
    for fstep in fh:
        step_vars = fstep.available_variables()

        for name, info in step_vars.items():
            print("variable_name: " + name)
            for key, value in info.items():
                print("\t" + key + ": " + value)
            print("\n")

The command above doesn't print anythin

When I try to read data directly from fh it works fine:

data = rh.read("a_par")

When I try reading the fields from fh in a loop I get an error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-61-094e1f12f241> in <module>
      2     fname = join(datapath, f"xgc.3d.{nn_idx:05d}.c.bp")
      3     with adios2.open(fname, "r") as fh:
----> 4         apar = fh.read("a_par")
      5         dpot = fh.read("dpot")
      6         fh.close()

ValueError: ERROR: offset 0 from steps start 41 in variable a_par is beyond the largest available step = 41, check Variable SetStepSelection argument stepsCount (random access), or number of BeginStep calls (streaming), in call to Get

How can I consistently read the data from my files?

Assignee
Assign to
Time tracking