Skip to content

WIP: return fortran arrays in original order in python high-level interface

Podhorszki, Norbert requested to merge github/fork/germasch/pr/layout into master

Created by: germasch

So this is what I originally set out to do:

  • adds a m_OriginalLayout member to core::Variable, which defaults to Unknown, but BP3 and BP4 engines will set it to the original layout the data was written with it.
  • bpls behavior is changed slightly (only): If the data was originally written from Fortran, it'll add the note (transposed) to the shape, but other than that, it still shows the dimensions reversed / dumps the data transposed. (This could be changed in the future to present the data how it was originally written, if that's considered more useful than current behavior)
  • The Fortran HL interface is changed to return arrays in their original index order. Nothing changes for arrays written in row-major, but arrays written as column-major will be read back in their original order. No transposing / rearranging of the data occurs, one just needs to tell numpy what the layout is.

More specifically, the read methods gain an additional order parameter. 'F' means ,interpret the data as column-major (so if it was written by Fortran, it'll show up in its original order. If it was originally row-major, this makes it show up transposed. 'C' means, interpret the data as row-major, which is what current behavior is always. 'K' means, interpret the data as the layout that it was saved as (if known, default row-major).

If no order parameter is provided, it defaults to 'K', which means both data written by C and Fortran will show up in their original order, which is what most people would want. However, it is a change in behavior. The default order, if not provided as an argument, can be easily changed using

import adios2
adios2.file.read_order = 'C'

will make restore current behavior. So while this PR changes behavior, it's at least easy to restore the original behavior if anyone turns out to be affected. I chose this way because the default behavior will avoid confusion, at the cost of potentially causing issues to someone relying on the old behavior. One could prioritize the opposite way, ie., keep old behavior by default and everyone who would like to see their Fortran arrays presented in original order would need to explicitly set read_order to 'K'.

Fixes #1665

Merge request reports