Debug hello_bpWriter

Bug fix in Collective metadata serialization using threads
Added more API functions in Python bindings

bug1: when writing value characteristic bpdump requires the length before the value
bug2: fixed merging serialized collective metadata index

examples/hello/bpWriter/hellBPPutDeferred.cpp

Passed ctest (MPI and nonMPI) excluding ADIOS1 engine, and Python bindings
HDF5, C bindings and C++ tests working
Introduced BeginStep EndStep functions
Renamed IO GetVariable/Attribute to InquireVariable/Attribute

Getting and defining available Variables at Open
Storing Variable position in Metadata, added to VariableBase
mpidummy: Added support for MPI_Reduce, MPI_Gatherv
Added utils and BPLS

Implemented basic buffer reading functionality
Added threading for bp serialization/deserialization
Refactored MPI functionality
Engine is now a reference
Initial Read API

First working examples
To do:
Resolve conflicts
Add tests after merging with current master

Bugs: 1) Missing binary mode in FilePointer fopen and FStream ofstream
transports
      2) Replacing reinterpret_cast of size_t to uint_64, not the same
on Windows 64-bit

Removed some warnings generated by Visual Studio

Added static_cast for type conversions

Refactor XML reader to restore original check on single engine
conditions and eliminate warnings. Test added.

Allow ctime and fopen "not secure" warnings: now using binary fstream as
default file IO library on Windows

Added test for generating profiling.json
Fixed inclusion for NLohmannJson json.hpp in Test
Tested resulting profiling.json on R (jsonlite), Python (json)
and C++ (json for modern C++)

Replace std::copy with std::memcpy in std::thread
Added threads to profiling.log
Limiting the use of threads

TO DO: large scale runs to enforce threads policy

Added example bpFlushWriter (incomplete).

To do:
Memory buffer reallocation strategy

bool flag checks are implicit
Removed single letter enums in ADIOSTypes.h
All enums are enum class in ADIOSTypes.h using CamelCase
ADIOS uses delegating constructor
Using return type of emplace for IO (in ADIOS) and Variable (in IO)
Removed all base class include from inl template file

1 Now IO (replaced Method) and is the factory for Variables and Engines
2 Reduced core components to user-public objects only
3 Moved and reorganized all helper functions to helper directory
4 Engines are now lightweight (except for ADIOS1Reader WIP) using MACRO
5 HF5Common and ADIOS1Common (except for Readers WIP) are now part of the toolkit so they can be reused by Engines
6 TransportMan is a new layer for transport management (file is default)
7 DataMan will be implemented under toolkit/transportman/dataman
8 Template separation (tcc and inl) applied all over the code
9 Improved Doxygen documentation

Runtime Issues:
DataMan library compilation (cacheman)
ADIOS1 Warning
TestADIOSInterfaceWrite catches exceptions

Created public function ResizeBuffer in BP1Writer

Adapted to the new code structure
BP1Writer is a self-contained class that can be reused across Engines
BP1Writer allows for threaded copies for large payloads

To do:
Data Buffer memory management strategy

Move adiosTemplates.h to adiosTemplate.inc

William F Godoy authored 7 years ago and Atkins, Charles Vernon committed 7 years ago

Changed BP1Writer.h by BPWriter.tcc in unique "public" header BP1.h

Moved more template functions in BP1Writer.h to BP1Writer.cpp
Added Macros in ADIOSMacros.h for handling overloaded template functions


To do:
Extend 1 arg template to 2 arg for template function in BP1Writer.tcc

BP1Writer.tcc is included at the end of BP1Writer.h and nowhere else.

Changed adiosTemplates.h and BP1 functions for STL's insert and copy to
a Buffer in sequential mode

Now meets the python dictionary format

helloBPWriter and timeBPWriter