Merge pull request #1039 from aronhelser/doc_tweaks

ADIOS

*****

The `adios2::ADIOS` component is the initial point between an application and the ADIOS2 library. Applications can be classified as MPI and non-MPI based. We start focusing on MPI applications as their non-MPI equivalent just removes the MPI-specific communicator. 

The `adios2::ADIOS` component is the initial contact point between an application and the ADIOS2 library. Applications can be classified as MPI and non-MPI based. We start by focusing on MPI applications as their non-MPI equivalent just removes the MPI-specific communicator.

.. code-block:: c++

.. caution::

   Running with the default debug mode (turned on) is highly recommended as it performs additional exception checks that the user can fix in case of an error or ill-formed object (std::invalid_argument). Turning the debug mode option should be used with caution when the user inputs to the resulting workflow is stable.

   Running with the default debug mode (turned on) is highly recommended as it performs additional exception checks that the user can fix in case of an error or ill-formed object (std::invalid_argument). Turning the debug mode option off should be used with caution, when the user inputs to the resulting workflow is stable.

.. note::

   Unexpected system failures and runtime errors are always checked by throwing std::runtime_error. Keep in mind that Segmentation Faults are NOT runtime exceptions. We try to keep user interactions as friendly as possible, please report if any bugs on github: https://github.com/ornladios/ADIOS2/issues  

   Unexpected system failures and runtime errors are always checked by throwing std::runtime_error. Keep in mind that Segmentation Faults are NOT runtime exceptions. We try to keep user interactions as friendly as possible, please report any bugs on github: https://github.com/ornladios/ADIOS2/issues

`adios2::ADIOS` objects can be created in MPI and non-MPI (serial) mode. Optionally, a runtime configuration file can be passed to the constructor indicating the full file path, name and extension. Thus resulting in:

    adios2::IO bpReader = adios.DeclareIO("BPReader");

This function returns a reference to an existing IO class object that lives inside the ADIOS object that created it. The ioName identifier input must be unique for each IO. Trying to declare an IO object with the same name twice will throw an exception if the debugMode is on in the ADIOS object constructor.

IO names are used to identify IO components in the runtime configuration file, `ref:Runtime Configuration Files` 

This function returns a reference to an existing IO class object that lives inside the ADIOS object that created it. The `ioName` identifier input must be unique for each IO. Trying to declare an IO object with the same name twice will throw an exception if the debugMode is on in the ADIOS object constructor.

IO names are used to identify IO components in the runtime configuration file, :ref:`Runtime Configuration Files`

As shown in the diagram below, each resulting IO object is self-managed and independent, thus providing an adaptable way to perform different kinds of I/O operations. Users must be careful not to create conflicts between system level unique I/O identifiers: file names, IP address and port, MPI Send/Receive message rank and tag, etc.

Engine Polymorphism has a two-fold goal:

1. Each Engine targets orthogonal IO scenarios targeting use cases (e.g. Files, WAN, InSitu MPI, etc) using a simple, unified API.

1. Each Engine implements an orthogonal IO scenario targeting a use case (e.g. Files, WAN, InSitu MPI, etc) using a simple, unified API.

2. Allow developers to build their own custom system solution based on their particular requirements in the own playground space. Resusable toolkit objects are available inside ADIOS2 for common tasks: bp buffering, transport management, transports, etc.

A class that extends the Engine must be thought of a solution to a range of IO applications. Each engine must provide a list of supported parameters, set in the IO object creating this engine in ``IO::SetParameters, IO::SetParameter``, and supported transports (and their parameters) in ``IO::AddTransport``. Each Engine particular options are documented in Chapter XX.

A class that extends Engine must be thought of as a solution to a range of IO applications. Each engine must provide a list of supported parameters, set in the IO object creating this engine using ``IO::SetParameters, IO::SetParameter``, and supported transports (and their parameters) in ``IO::AddTransport``. Each Engine's particular options are documented in :ref:`Supported Engines`.

.. note::

   ``adios2::Params`` is an alias to ``std::map<std::string,std::string>`` to pass parameters as key-value string pairs

   ``adios2::Params`` is an alias to ``std::map<std::string,std::string>`` to pass parameters as key-value string pairs, which can be initialized with curly-brace initializer lists.

.. code-block:: c++

Defining, Inquiring and Removing Variables and Attributes

---------------------------------------------------------

The template functions DefineVariable<T> allows subscribing self-describing data into ADIOS2 by returning a reference to a Variable class object whose scope is the same as the IO object that created it. The user must provide a unique name (among Variables), the dimensions: MPI global: shape, MPI local: start and offset, optionally a flag indicating that dimensions are to know to be constant and a data pointer if defined in the application. Note: actual data is not passed at this stage. This is done by the Engine functions Put/Get for Variables. See the :ref:`Variable` section for supported types and shapes.

The template functions ``DefineVariable<T>`` allows subscribing self-describing data into ADIOS2 by returning a reference to a Variable class object whose scope is the same as the IO object that created it. The user must provide a unique name (among Variables), the dimensions: MPI global: shape, MPI local: start and offset, optionally a flag indicating that dimensions are known to be constant, and a data pointer if defined in the application. Note: actual data is not passed at this stage. This is done by the Engine functions Put/Get for Variables. See the :ref:`Variable` section for supported types and shapes.

.. tip::

   ``adios2::Dims`` is an alias to ``std::vector<std::size_t>``, while ``adios2::ConstantDims`` is an alias to bool ``true``. Use them for code clarity.

                                  const size_t elements);

In situations in which a variable and attribute has been previously defined:

1) a variable/attribute reference goes out of scope, or 2) when reading from an incoming stream, IO can inquire the current variables and attributes and return a pointer acting as reference. If the inquired variable/attribute is not found, then the pointer reference is nullptr. 

1) a variable/attribute reference goes out of scope, or 2) when reading from an incoming stream, IO can inquire the current variables and attributes and return a pointer acting as reference. If the inquired variable/attribute is not found, then ``nullptr`` is returned.

.. code-block:: c++

.. note::

   The reason for returning a pointer when inquiring, unlike references when defining, is because nullptr is a valid state (e.g. variables hasn't arrived in a stream, wasn't previously defined or wasn't written in a file). 

   The reason for returning a pointer when inquiring, unlike references when defining, is because ``nullptr`` is a valid state (e.g. variables hasn't arrived in a stream, wasn't previously defined or wasn't written in a file).

.. tip::  

   It is good practice to check for nullptr in the pointer returned by InquireVariable<T> or InquireAttribute<T> 

   Always check for ``nullptr`` in the pointer returned by ``InquireVariable<T>`` or ``InquireAttribute<T>``

.. caution::

   Since Inquire are template functions, name and type must match to those of a variable/attribute to look for.

   Since Inquire are template functions, name and type must both match the variable/attribute you are looking for.

Removing Variables and Attributes can be done in a on-by-one basis or by removing all existing variables or attributes in IO.

Removing Variables and Attributes can be done one at a time or by removing all existing variables or attributes in IO.

.. code-block:: c++

.. tip::

   It is good practice to check the bool flag returned by RemoveVariable or RemoveAttribute

   It is good practice to check the bool flag returned by RemoveVariable or RemoveAttribute.

Opening an Engine

    /** Signatures */

    /** Provide a new MPI communicator other than from ADIOS->IO->Engine */

    adios2::Engine adios2::IO::Open( const std::string &name, 

    adios2::Engine &adios2::IO::Open( const std::string &name,

                                      const adios2::Mode mode,

                                      MPI_Comm mpiComm );

    /** Reuse the MPI communicator from ADIOS->IO->Engine \n or non-MPI serial mode */

    adios2::Engine adios2::IO::Open(const std::string &name, 

    adios2::Engine &adios2::IO::Open(const std::string &name,

                                     const adios2::Mode mode);

1. Data compression/decompression, lossy and lossless.

2. Callback functions (C++11 bindings only) supported by specific engines

ADIOS2 enables the use of third-party libraries to carry these tasks.

ADIOS2 enables the use of third-party libraries to execute these tasks.

.. warning::

   Make sure you ADIOS2 library installation used for writing and reading was linked with a compatible version of a third-party dependency when working with operators. ADIOS2 will issue an exception if an operator library dependency is missing.      

 No newline at end of file

   Make sure your ADIOS2 library installation used for writing and reading was linked with a compatible version of a third-party dependency when working with operators. ADIOS2 will issue an exception if an operator library dependency is missing.

 Python         objects/member functions    numpy arrays.

============== ========================== ==================================

The following section provides a common overview to all languages based on the C++11 APIs. For each specific language go to the `ref:Language Bindings APIs` section, but it's highly recommended to read this section as components map 1-to-1 in other languages. 

The following section provides a common overview to all languages based on the C++11 APIs. For each specific language go to the :ref:`Full Language Bindings APIs` section, but it's highly recommended to read this section as components map 1-to-1 in other languages.

The following figure depicts the components hierarchy from the application's point of view.

* **Variable**: Variables are the link between self-describing representation in the ADIOS2 library and data from applications. Variables are identified by unique names in the scope of the particular IO that created them. When the Engine API functions are called, a Variable must be provided along with the application data.

* **Attribute**: an Attributes adds extra information to the overall variables dataset defined in the IO class. They can be single or array values.

* **Attribute**: Attributes add extra information to the overall variables dataset defined in the IO class. They can be single or array values.

* **Engine**: Engines define the actual system executing the heavy IO tasks at Open, BeginStep, Put, Get, EndStep and Close. Due to polymorphism, new IO system solutions can be developed quickly reusing internal components and reusing the same API. The default engine, if IO.SetEngine is not called, is the binary-pack bp file reader and writer: BPFile.

* **Engine**: Engines define the actual system executing the heavy IO tasks at Open, BeginStep, Put, Get, EndStep and Close. Due to polymorphism, new IO system solutions can be developed quickly reusing internal components and reusing the same API. If IO.SetEngine is not called, the default engine is the binary-pack bp file reader and writer: **BPFile**.

* **Operator**: (under development) this component defines possible operations to be applied on adios2 self-describing data. This higher level abstraction is needed to provide support to: Callback functions, Transforms, Analytics funcions, Data Models functionality, etc. Any required task will be executed within the Engine. One or many operators can be associated with any of the adios2 objects or a group of them.

* **Operator**: (under development) this component defines possible operations to be applied on adios2 self-describing data. This higher level abstraction is needed to provide support for: Callback functions, Transforms, Analytics funcions, Data Models functionality, etc. Any required task will be executed within the Engine. One or many operators can be associated with any of the adios2 objects or a group of them.