Python docs progress in simple API (f2087dc9) · Commits · Podhorszki, Norbert / ADIOS2

bindings/Python/py11glue.cpp

+49 −14

Original line number	Diff line number	Diff line
		@@ -416,7 +416,23 @@ PYBIND11_MODULE(adios2, m)
		adios2::py11::File::Read,
		pybind11::return_value_policy::take_ownership,
		pybind11::arg("name"), pybind11::arg("start"),
		pybind11::arg("count"))
		pybind11::arg("count"), R"md(
		Reads a selection piece in dimension for current step
		(streaming mode step by step)

		Parameters
		name
		variable name
		start
		variable local offset selection
		count
		variable local dimension selection from start

		Returns
		array: numpy array
		values of variable name for current step
		empty if exception is thrown
		)md")

		.def("read", (pybind11::array (adios2::py11::File::*)(
		const std::string &, const adios2::Dims &,
		@@ -426,24 +442,43 @@ PYBIND11_MODULE(adios2, m)
		pybind11::arg("name"), pybind11::arg("start"),
		pybind11::arg("count"), pybind11::arg("stepstart"),
		pybind11::arg("stepcount"), R"md(
		Random access read, only valid with File Engines

		Args:
		name: variable name to read.
		start: variable offset dimensions.
		count: variable local dimensions from offset.
		stepstart: variable step start
		stepcount: variable number of steps to read
		Random access read allowed to select steps,
		only valid with File Engines

		Parameters

		name
		variable to be read

		start
		variable offset dimensions

		count
		variable local dimensions from offset

		Returns:
		array (numpy) resulting array from selection
		stepstart
		variable step start

		stepcount
		variable number of steps to read

		Returns

		array
		resulting array from selection
		)md")

		.def("close", &adios2::py11::File::Close, R"md(
		closes file
		Closes file, thus becoming unreachable.
		Not required if using open in a with-as statement.
		Required in all other cases per-open to avoid resource leaks.
		)md")

		.def("currentstep", &adios2::py11::File::CurrentStep, R"md(
		:return: current step
		Return current step when using for-in loops, read mode only
		Returns

		current step
		)md");
		}

docs/user_guide/source/api_full/python.rst

+7 −1

Original line number	Diff line number	Diff line
		@@ -2,4 +2,10 @@
		Python bindings
		***************

		Here goes the full Python API
		.. note::

		Product Application Developers targeting finer-level control for their IO tasks for optimization should use the current full APIs. If you want to use ADIOS2 in simple use cases (e.g. reading a file for analysis, interactive Python, or saving some data for a small project) please refer to the :ref:`Simple Language Bindings APIs` for a flat learning curve.

		The full Python APIs follow very closely the full C++11 API interface.
		Work in progress.

docs/user_guide/source/api_simple/python.rst

+12 −6

Original line number	Diff line number	Diff line
		@@ -2,12 +2,10 @@
		Python simple bindings
		**********************

		.. autoclass:: adios2::File
		:members:

		Python simple bindings follow closely the :ref:`C++ simple bindings`. Just like the full APIs, the rely on numpy and (optionally) on mpi4py.

		Write example:
		Write example
		-------------

		.. code-block:: python

		@@ -38,7 +36,9 @@ Write example:
		# advances to next step
		fw.write("pressure", pressure, shape, start, count, endl=True)

		Read "stepping/streaming" example:

		Read "stepping/streaming" example
		---------------------------------

		.. code-block:: python

		@@ -69,3 +69,9 @@ Read "stepping/streaming" example:
		temperature = fh_step.read("temperature", start, count)
		pressure = fh_step.read("pressure", start, count)


		File class API
		--------------

		.. autoclass:: adios2::File
		:members:
		No newline at end of file

docs/user_guide/source/engines/bpfile.rst

+13 −10

Original line number	Diff line number	Diff line
		@@ -30,29 +30,32 @@ This engine allows the user to fine tune the buffering operations through the fo

		1. Profile: turns ON/OFF profiling information right after a run

		2. CollectiveMetadata: turns ON/OFF forming collective metadata during run (used by large scale HPC applications)
		2. ProfileUnits: set profile units according to the required measurement scale for intensive operations

		3. Threads: number of threads provided from the application for buffering, use this for very large variables
		3. CollectiveMetadata: turns ON/OFF forming collective metadata during run (used by large scale HPC applications)

		4. InitialBufferSize: initial memory provided for buffering (minimum is 16Kb)
		4. Threads: number of threads provided from the application for buffering, use this for very large variables

		5. BufferGrowthFactor: exponential growth factor for initial buffer > 1, default = 1.05.
		5. InitialBufferSize: initial memory provided for buffering (minimum is 16Kb)

		6. MaxBufferSize: maximum allowable buffer size (must be larger than 16Kb). If to large adios2 will throw an exception.
		6. BufferGrowthFactor: exponential growth factor for initial buffer > 1, default = 1.05.

		7. FlushStepsCount: user can select how often to produce the more expensive collective metadata file in terms of steps: default is 1. Increase to reduce adios2 collective operations footprint, with the trade-off of reducing checkpoint frequency. Buffer size will increase until first steps count if MaxBufferSize is not set.
		7. MaxBufferSize: maximum allowable buffer size (must be larger than 16Kb). If to large adios2 will throw an exception.

		==================== ===================== ==============================
		8. FlushStepsCount: user can select how often to produce the more expensive collective metadata file in terms of steps: default is 1. Increase to reduce adios2 collective operations footprint, with the trade-off of reducing checkpoint frequency. Buffer size will increase until first steps count if MaxBufferSize is not set.

		==================== ===================== =========================================================
		Key Value Format Default and Examples
		==================== ===================== ==============================
		==================== ===================== =========================================================
		Profile string On/Off On, Off
		ProfileUnits string Microseconds, Milliseconds, Seconds, Minutes, Hours
		CollectiveMetadata string On/Off On, Off
		Threads integer > 1 1, 2, 3, 4, 16, 32, 64
		InitialBufferSize float+units >= 16Kb 16Kb, 10Mb, 0.5Gb
		MaxBufferSize float+units >= 16Kb at EndStep, 10Mb, 0.5Gb
		BufferGrowthFactor float > 1 1.05, 1.01, 1.5, 2
		FlushStepsCount integer > 1 1 5, 1000, 50000
		==================== ===================== ==============================
		FlushStepsCount integer > 1 1, 5, 1000, 50000
		==================== ===================== =========================================================


		Only file transport types are supported. Optional parameters for `IO::AddTransport` or in runtime config file transport field:

Original line number	Diff line number	Diff line
		@@ -30,29 +30,32 @@ This engine allows the user to fine tune the buffering operations through the fo

		1. Profile: turns ON/OFF profiling information right after a run

		2. CollectiveMetadata: turns ON/OFF forming collective metadata during run (used by large scale HPC applications)
		2. ProfileUnits: set profile units according to the required measurement scale for intensive operations

		3. Threads: number of threads provided from the application for buffering, use this for very large variables
		3. CollectiveMetadata: turns ON/OFF forming collective metadata during run (used by large scale HPC applications)

		4. InitialBufferSize: initial memory provided for buffering (minimum is 16Kb)
		4. Threads: number of threads provided from the application for buffering, use this for very large variables

		5. BufferGrowthFactor: exponential growth factor for initial buffer > 1, default = 1.05.
		5. InitialBufferSize: initial memory provided for buffering (minimum is 16Kb)

		6. MaxBufferSize: maximum allowable buffer size (must be larger than 16Kb). If to large adios2 will throw an exception.
		6. BufferGrowthFactor: exponential growth factor for initial buffer > 1, default = 1.05.

		7. FlushStepsCount: user can select how often to produce the more expensive collective metadata file in terms of steps: default is 1. Increase to reduce adios2 collective operations footprint, with the trade-off of reducing checkpoint frequency. Buffer size will increase until first steps count if MaxBufferSize is not set.
		7. MaxBufferSize: maximum allowable buffer size (must be larger than 16Kb). If to large adios2 will throw an exception.

		==================== ===================== ==============================
		8. FlushStepsCount: user can select how often to produce the more expensive collective metadata file in terms of steps: default is 1. Increase to reduce adios2 collective operations footprint, with the trade-off of reducing checkpoint frequency. Buffer size will increase until first steps count if MaxBufferSize is not set.

		==================== ===================== =========================================================
		Key Value Format Default and Examples
		==================== ===================== ==============================
		==================== ===================== =========================================================
		Profile string On/Off On, Off
		ProfileUnits string Microseconds, Milliseconds, Seconds, Minutes, Hours
		CollectiveMetadata string On/Off On, Off
		Threads integer > 1 1, 2, 3, 4, 16, 32, 64
		InitialBufferSize float+units >= 16Kb 16Kb, 10Mb, 0.5Gb
		MaxBufferSize float+units >= 16Kb at EndStep, 10Mb, 0.5Gb
		BufferGrowthFactor float > 1 1.05, 1.01, 1.5, 2
		FlushStepsCount integer > 1 1 5, 1000, 50000
		==================== ===================== ==============================
		FlushStepsCount integer > 1 1, 5, 1000, 50000
		==================== ===================== =========================================================


		Only file transport types are supported. Optional parameters for `IO::AddTransport` or in runtime config file transport field:

Admin message