Commit d0dc5a3b authored by William F Godoy's avatar William F Godoy
Browse files

Adding visualization docs

parent addc344b
......@@ -126,11 +126,13 @@ exclude_patterns = [
'api_high/python.rst',
'api_high/cxx11.rst',
# 'api_high/api_high.rst',
'utilities/adios2-config.rst',
'utilities/adios_reorganize.rst',
# 'utilities/utilities.rst',
'utilities/bpls.rst',
'utilities/sst_conn_tool.rst',
# 'ecosystem/utilities.rst',
'ecosystem/utilities/adios2-config.rst',
'ecosystem/utilities/adios_reorganize.rst',
'ecosystem/utilities/bpls.rst',
'ecosystem/utilities/sst_conn_tool.rst',
'ecosystem/visualization.rst'
'ecosystem/visualization/vtk.rst'
'engines/dataman.rst',
'engines/insitu_mpi.rst',
'engines/hdf5.rst',
......
......@@ -17,7 +17,7 @@ Currently supported tools are:
* ``adios2-config``
* ``sst_conn_tool`` : SST staging engine connectivity diagnostic tool
.. include:: bpls.rst
.. include:: adios_reorganize.rst
.. include:: adios2-config.rst
.. include:: sst_conn_tool.rst
.. include:: utilities/bpls.rst
.. include:: utilities/adios_reorganize.rst
.. include:: utilities/adios2-config.rst
.. include:: utilities/sst_conn_tool.rst
*************
adios2-config
*************
`adios2-config` is the preferred to use in non-CMake (*e.g.* Makefile builds).
Running the `adios2-config` command under `adios2-install-dir/bin/adios2-config` will generate the following usage information:
.. code-block:: bash
./adios2-config --help
adios2-config (--help | [--c-flags] [--c-libs] [--cxx-flags] [--cxx-libs] [-fortran-flags] [--fortran-libs])
--help Display help information
-c Both compile and link flags for the C bindings
--c-flags Preprocessor and compile flags for the C bindings
--c-libs Linker flags for the C bindings
-x Both compile and link flags for the C++ bindings
--cxx-flags Preprocessor and compile flags for the C++ bindings
--cxx-libs Linker flags for the C++ bindings
-f Both compile and link flags for the F90 bindings
--fortran-flags Preprocessor and compile flags for the F90 bindings
--fortran-libs Linker flags for the F90 bindings
Please refer to the :ref:`From non-CMake build systems` for more information on how to use this command.
\ No newline at end of file
################
Visualizing Data
################
.. note::
As of version 2.4.0 this work should be treated as experimental. As it builds on top of the library in third party ecosystems, and doesn't ship with ADIOS 2 source code.
Certain ADIOS 2 bp files, and in-memory streams in the future), can be recognized by third party products to enable data visualization. The expectation is to keep adding support to the list of products enabling ADIOS 2 for visualization purposes. The goal of this section is to describe the currently cover cases and how to create an ADIOS 2 BP file to accomodate the visualization product requirements.
.. include:: visualization/vtk.rst
**********************
Using VTK and Paraview
**********************
ADIOS streams can now be seamlessly integrated into the well-established visualization and analysis `Visualization Toolkit <https://vtk.org/>`_ (VTK) and `Paraview <https://www.paraview.org/>`_ products. Data sets can be described with an additional attribute that conforms to the well known and documented `VTK XML data model formats <https://vtk.org/wp-content/uploads/2015/04/file-formats.pdf>`_ as a high-level descriptors that will allow interpretation of ADIOS 2 variables into a hierarchy understood by the VTK infrastructure. This XML data format is saved in ADIOS as either an attribute, and optionally, as an additional `vtk.xml` file inside a resulting bp data set `file.bp.dir/vtk.xml` from the default engine, see :ref:`BP3 (Default)`.
.. note::
This is work in progress developed inside VTK, not in the ADIOS 2 code base
Current limitations:
* Only works with MPI builds of VTK and Paraview
* Support only one Block per ADIOS dataset
* Only supports BP Files, streams will be added in the future
* Currently working up to 3D (and linearized 1D) variables for scalars and vectors.
* Image Data, vti, is supported with ADIOS2 Global Array Variables only
* Unstructured Grid, vtu, is supported with ADIOS2 Local Arrays Variables only
So far two "VTK File" types are supported:
1. Image data (.vti)
2. Unstructured Grid (.vtu)
The main idea is to populate the above XML format contents describing the extent and the data arrays with ADIOS variables in the BP data set. The result is a more-than-compact typical VTK data file since extra information about the variable, such as dimensions and type, as they already exist in the BP data set.
A typical VTK image data XML descriptor (.vti):
.. code-block:: xml
<?xml version="1.0"?>
<VTKFile type="ImageData">
<ImageData WholeExtent="x1 x2 y1 y2 z1 z2" Origin="x0 y0 z0" Spacing="dx dy dz">
<Piece Extent="x1 x2 y1 y2 z1 z2">
<PointData>
<DataArray Name="p1"/>
<DataArray Name="p2"/>
</PointData>
<CellData>
<DataArray Name="c1"/>
<DataArray Name="c2"/>
</CellData>
</Piece>
</ImageData>
</VTKFile>
A typical VTK unstructured grid XML descriptor (.vtu):
.. code-block:: xml
<?xml version="1.0"?>
<VTKFile type="ImageData">
<ImageData WholeExtent="x1 x2 y1 y2 z1 z2" Origin="x0 y0 z0" Spacing="dx dy dz">
<Piece Extent="x1 x2 y1 y2 z1 z2">
<PointData>
<DataArray Name="p1"/>
<DataArray Name="p2"/>
</PointData>
<CellData>
<DataArray Name="c1"/>
<DataArray Name="c2"/>
</CellData>
</Piece>
</ImageData>
</VTKFile>
In addition, VTK can interpret physical-time or output-step varying data stored with ADIOS by resusing the special "TIME" tag. This is better illustrated in the following section.
Saving the vtk.xml data model
-----------------------------
For the full source code of the following illustration example see the `gray-scott adios2 tutorial <https://github.com/pnorbert/adiosvm/tree/master/Tutorial/gray-scott>`_
To incorporate the data model in a BP data file, the application has two options:
1) Adding a string attribute called "vtk.xml" in code. "TIME" is a special tag for adding physical time variables.
.. code-block:: c++
const std::string imageData = R"(
<?xml version="1.0"?>
<VTKFile type="ImageData" version="0.1" byte_order="LittleEndian">
<ImageData WholeExtent=")" + extent + R"(" Origin="0 0 0" Spacing="1 1 1">
<Piece Extent=")" + extent + R"(">
<CellData Scalars="U">
<DataArray Name="U" />
<DataArray Name="V" />
<DataArray Name="TIME">
step
</DataArray>
</CellData>
</Piece>
</ImageData>
</VTKFile>)";
io.DefineAttribute<std::string>("vtk.xml", imageData);
.. tip::
C++11 users should take advantage of the added string literal capabilities in C++11, R''( )", as in the example above.
The resulting bpls output should contain the "vtk.xml" attribute and the variables in the model:
.. code-block:: bash
> bpls gs.bp -lav
File info:
of variables: 3
of attributes: 7
statistics: Min / Max
double Du attr = 0.2
double Dv attr = 0.1
double F attr = 0.02
double U 24*{48, 48, 48} = 0.107439 / 1.04324
double V 24*{48, 48, 48} = 0 / 0.672232
double dt attr = 1
double k attr = 0.048
double noise attr = 0.01
int32_t step 24*scalar = 0 / 575
string vtk.xml attr =
<VTKFile type="ImageData" version="0.1" byte_order="LittleEndian">
<ImageData WholeExtent="0 49 0 49 0 49" Origin="0 0 0" Spacing="1 1 1">
<Piece Extent="0 49 0 49 0 49">
<CellData Scalars="U">
<DataArray Name="U" />
<DataArray Name="V" />
<DataArray Name="TIME">
step
</DataArray>
</CellData>
</Piece>
</ImageData>
</VTKFile>
2) Saving a "vtk.xml" file inside the file.bp.dir to describe the data after is created
.. code-block::
> cat gs.bp.dir/vtk.xml
<?xml version="1.0"?>
<VTKFile type="ImageData" version="0.1" byte_order="LittleEndian">
<ImageData WholeExtent=")" + extent + R"(" Origin="0 0 0" Spacing="1 1 1">
<Piece Extent=")" + extent + R"(">
<CellData Scalars="U">
<DataArray Name="U" />
<DataArray Name="V" />
<DataArray Name="TIME">
step
</DataArray>
</CellData>
</Piece>
</ImageData>
</VTKFile>
The result is that the generated BP file should be recognize by a branch of Paraview/VTK that must be built from source:
.. image:: http://i66.tinypic.com/33krfh2.png : alt: my-picture2
Similarly, unstructured grid (.vtu) support can be added with the limitations of using specific labels for the variable names setting the "connectivity", "vertices", and cell "types".
The following example is taken from example 2 of the `MFEM product examples website <https://mfem.org/examples/>`_ using ADIOS 2:
The resulting `bpls` output for unstructured grid data types:
.. code-block:: bash
File info:
of variables: 6
of attributes: 4
statistics: Min / Max
uint32_t NumOfElements {4} = 1024 / 1024
uint32_t NumOfVertices {4} = 1377 / 1377
string app attr = "MFEM"
uint64_t connectivity [4]*{1024, 9} = 0 / 1376
uint32_t dimension attr = 3
string glvis_mesh_version attr = "1.0"
double sol [4]*{1377, 3} = -0.201717 / 1.19304
uint32_t types scalar = 11
double vertices [4]*{1377, 3} = -1.19304 / 8.20172
string vtk.xml attr =
<VTKFile type="UnstructuredGrid" version="0.1" byte_order="LittleEndian">
<UnstructuredGrid>
<Piece NumberOfPoints="NumOfVertices" NumberOfCells="NumOfElements">
<Points>
<DataArray Name="vertices" />
</Points>
<Cells>
<DataArray Name="connectivity" />
<DataArray Name="types" />
</Cells>
<PointData>
<DataArray Name="sol" />
</PointData>
</Piece>
</UnstructuredGrid>
</VTKFile>
and resulting visualization in Paraview for different "cell" types:
.. image:: http://i67.tinypic.com/sll8gl.png : alt: my-picture3
Build VTK and Paraview with ADIOS 2 Support
-------------------------------------------
.. note::
Currently the implementation for ADIOS 2 readers exist in VTK and Paraview branches. We expect this to be part of the VTK and Paraview release cycle with their upcoming releases. Users must build from source and point to these branches until formal merge into their master branches is done.
Paraview and its VTK dependency must be built with the following CMake options:
1. `-DVTK_MODULE_ENABLE_VTK_IOADIOS2=YES`
2. `-DVTK_USE_MPI=ON`
3. `-DPARAVIEW_USE_MPI=ON`
For comprehensive build instructions see the documentation for `VTK <https://vtk.org/>`_ (VTK) and `Paraview <https://www.paraview.org/>`_ , respectively.
We look forward to maintain and expand the current service capabilities developed in VTK and Paraview to support more complex scenarios: e.g. moving mesh, multigrid, real-time streams, as well as enriching the data model capabilities base on VTK XML formats.
......@@ -10,24 +10,18 @@ Funded by the `Exascale Computing Project (ECP) <https://www.exascaleproject.org
.. toctree::
:caption: Introduction
:titlesonly:
:maxdepth: 1
introduction/introduction
.. toctree::
:caption: Setting Up
:titlesonly:
:maxdepth: 1
setting_up/setting_up
.. toctree::
:caption: Basics
:titlesonly:
:maxdepth: 1
components/components
engines/engines
......@@ -37,16 +31,13 @@ Funded by the `Exascale Computing Project (ECP) <https://www.exascaleproject.org
.. toctree::
:caption: Ecosystem Tools
:titlesonly:
:maxdepth: 1
utilities/utilities
ecosystem/utilities
ecosystem/visualization
.. toctree::
:caption: Additional Information
:titlesonly:
:maxdepth: 1
faq/faq
advice/advice
......
****************************************
Building, Testing, and Installing ADIOS2
****************************************
*****************************************
Building, Testing, and Installing ADIOS 2
*****************************************
To build ADIOS2, clone the repository and invoke the canonical CMake build sequence:
.. caution::
Always do a fresh build from scratch if your source is updated with considerable changes *e.g.* `git pull`
To build ADIOS 2, clone the repository and invoke the canonical CMake build sequence:
.. code-block:: bash
......
*************
adios2-config
*************
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment