Commit c3db52da authored by Bolea Sanchez, Vicente Adolfo's avatar Bolea Sanchez, Vicente Adolfo Committed by Kitware Robot
Browse files

Merge topic 'update-to-1.6.0-rc1'

19c1b99f 1.6.0-rc1 is our 7th official release of VTK-m.
4e94d830 Add release notes for v1.6.0
9d345733 Merge commit 'd2d1c854' into update-to-1.6.0-rc1
74ffad9b 1.5.1 is a bugfix release for VTK-m.
f67dcfc3 Add release notes for v1.5.1
38af6dbf add Particle.h and Deprecated.h to the installed headers
d5f50c5a 3b7b21c8 Do not use std::is_trivially_copyable on GCC 4.X

 Variant as trivially copyable
Acked-by: default avatarKitware Robot <>
Acked-by: default avatarKenneth Moreland <>
Merge-request: !2457
parents b00f5b44 19c1b99f
VTK-m 1.5.1 Release Notes
# Table of Contents
1. [Core](#Core)
- Variant trivial copy
2. [Build](#Build)
- GCC 4 trivially copyable
- MSVC flag fix
- GCC openmp workaround fix
- Correct gcc 9 warnings
- GCC 48 iterator fixes
- Aligned union check handle gcc 485
- Update check for aligned union
- Intel fix
- Correct msvc 2015 failure
- GCC 61 ice openmp and optimizations
- No deprecated nvcc vs
# Core
## Variant trivial copy
The Variant template can hold any type. If it is holding a type that is
non-copyable, then it has to make sure that appropriate constructors,
copiers, movers, and destructors are called.
Previously, these were called even the Variant was holding a trivially
copyable class because no harm no foul. If you were holding a trivially
copyable class and did a `memcpy`, that work work, which should make it
possible to copy between host and device, right?
In theory yes, but in practice no. The problem is that _Cuda_ is
outsmarting the code. It is checking that Variant is not trivially-
copyable by C++ semantics and refusing to push it.
So, change Variant to check to see if all its supported classes are
trivially copyable. If they are, then it use the default constructors,
destructors, movers, and copiers so that C++ recognizes it as trivially
7518d067 Try to fix uninitialized anonymous variable warning
5b18ffd7 Register Variant as trivially copyable if possible
16305bd8 Add tests of ArrayHandleMultiplexer on multiple devices
# Build
## GCC-4 trivially copyable
Although GCC 4.8 and 4.9 claim to be C++11 compliant, there are a few
C++11 features they do not support. One of these features is
`std::is_trivially_copyable`. So on these platforms, do not attempt to use
it. Instead, treat nothing as trivially copyable.
3b7b21c8 Do not use std::is_trivially_copyable on GCC 4.X
## Fix MSVC flag
Fix MSVC flags for CUDA builds.
07b55a95 Fix MSVC flags for CUDA builds.
## GCC OpenMP workaround fix
There is some behavior of GCC compilers before GCC 9.0 that is
incompatible with the specification of `OpenMP` 4.0. The workaround was
using the workaround any time a GCC compiler >= 9.0 was used. The proper
behavior is to only use the workaround when the GCC compiler is being
used and the version of the compiler is less than 9.0.
Also, switch to using `VTKM_GCC` to check for the GCC compiler instead of
GNUC. The problem with using GNUC is that many other compilers
pretend to be GCC by defining this macro, but in cases like compiler
workarounds it is not accurate.
033dfe55 Only workaround incorrect GCC behavior for OpenMP on GCC
## Correct GCC-9 warnings
870bd1d1 Removed unnecessary increment and decrement from ZFPDecode
f9860b84 Correct warnings found by gcc-9 in vtkm::Particle
## GCC-4.8 iterator fixes
The draft C++11 spec that GCC-4.X implemented against had some
defects that made implementing `void_t<...>` tricky.
83d4d4e4 ArrayPortalToIterators now compiles with GCC-4.X
## Aligned union check handle GCC
b36846e4 UnitTestVariant uses VTKM_USING_GLIBCXX_4
cbf20ac3 Merge branch 'upstream-diy' into aligned_union_check_handle_gcc_485
ac1a23be diy 2019-12-17 (bb86e1f7)
201e5c81 Add the gcc 4.8.5 release date to our VTKM_USING_GLIBCXX_4 check
## Update check for aligned union
Fixes #447 (closed)
This uses a more robust set of checks to determine if `std::aligned_union`
and `std::is_trivially_copyable` exist given the `libstdc++` version value
2e48d98d Merge branch 'upstream-diy' into update_check_for_aligned_union
bbd5db31 diy 2019-12-16 (e365b66a)
269261b9 Handle compiling against 4.X versions of libstdc++
## Intel fix
b6b20f08 Use brigand integer sequences on icc.
## Correct MSVC 2015 failure
f89672b7 UnitTestFetchArrayTopologyMapIn now compiles with VS2015
## GCC-61 ice OpenMP and optimizations
dc86ac20 Avoid a GCC 6.1 compiler regression that occurs when openmp is enabled
## No deprecated NVCC VS
The NVCC compiler under visual studio seems to give the error attribute does
not apply to any entity when you try to use the [[deprecated]] attribute.
So disable for this compiler configuration.
fb01d38a Disable deprecated attribute when using nvcc under VS
VTK-m 1.6 Release Notes
# Table of Contents
1. [Core](#Core)
- Add Kokkos backend
- Deprecate `DataSetFieldAdd`
- Move VTK file readers and writers into vtkm_io
- Remove VTKDataSetWriter::WriteDataSet just_points parameter
- Added VecFlat class
- Add a vtkm::Tuple class
- DataSet now only allows unique field names
- Result DataSet of coordinate transform has its CoordinateSystem changed
- `vtkm::cont::internal::Buffer` now can have ownership transferred
- Configurable default types
2. [ArrayHandle](#ArrayHandle)
- Shorter fancy array handle classnames
- `ReadPortal().Get(idx)`
- Precompiled `ArrayCopy` for `UnknownArrayHandle`
- Create `ArrayHandleOffsetsToNumComponents`
- Recombine extracted component arrays from unknown arrays
- UnknownArrayHandle and UncertainArrayHandle for runtime-determined types
- Support `ArrayHandleSOA` as a "default" array
- Removed old `ArrayHandle` transfer mechanism
- Order asynchronous `ArrayHandle` access
- Improvements to moving data into ArrayHandle
- Deprecate ArrayHandleVirtualCoordinates
- ArrayHandleDecorator Allocate and Shrink Support
- Portals may advertise custom iterators
- Redesign of ArrayHandle to access data using typeless buffers
- `ArrayRangeCompute` works on any array type without compiling device code
- Implemented ArrayHandleRandomUniformBits and ArrayHandleRandomUniformReal
- Extract component arrays from unknown arrays
- `ArrayHandleGroupVecVariable` holds now one more offset.
3. [Control Environment](#Control-Environment)
- Algorithms for Control and Execution Environments
4. [Execution Environment](#Execution-Environment)
- Scope ExecObjects with Tokens
- Masks and Scatters Supported for 3D Scheduling
- Virtual methods in execution environment deprecated
- Deprecate Execute with policy
5. [Worklets and Filters](#Worklets-and-Filters)
- Enable setting invalid value in probe filter
- Avoid raising errors when operating on cells
- Add atomic free functions
- Flying Edges
- Filters specify their own field types
6. [Build](#Build)
- Disable asserts for CUDA architecture builds
- Disable asserts for HIP architecture builds
7. [Other](#Other)
- Porting layer for future std features
- Removed OpenGL Rendering Classes
- Reorganization of `io` directory
- Implemented PNG/PPM image Readers/Writers
- Updated Benchmark Framework
- Provide scripts to build Gitlab-ci workers locally
- Replaced `vtkm::ListTag` with `vtkm::List`
- Add `ListTagRemoveIf`
- Write uniform and rectilinear grids to legacy VTK files
8. [References](#References)
# Core
## Add Kokkos backend
Adds a new device backend `Kokkos` which uses the kokkos library for parallelism.
User must provide the kokkos build and Vtk-m will use the default configured execution
## Deprecate `DataSetFieldAdd`
The class `vtkm::cont::DataSetFieldAdd` is now deprecated.
Its methods, `AddPointField` and `AddCellField` have been moved to member functions
of `vtkm::cont::DataSet`, which simplifies many calls.
For example, the following code
vtkm::cont::DataSetFieldAdd fieldAdder;
fieldAdder.AddCellField(dataSet, "cellvar", values);
would now be
dataSet.AddCellField("cellvar", values);
## Move VTK file readers and writers into vtkm_io
The legacy VTK file reader and writer were created back when VTK-m was a
header-only library. Things have changed and we now compile quite a bit of
code into libraries. At this point, there is no reason why the VTK file
reader/writer should be any different.
Thus, `VTKDataSetReader`, `VTKDataSetWriter`, and several supporting
classes are now compiled into the `vtkm_io` library. Also similarly updated
`BOVDataSetReader` for good measure.
As a side effect, code using VTK-m will need to link to `vtkm_io` if they
are using any readers or writers.
## Remove VTKDataSetWriter::WriteDataSet just_points parameter
In the method `VTKDataSetWriter::WriteDataSet`, `just_points` parameter has been
removed due to lack of usage.
The purpose of `just_points` was to allow exporting only the points of a
DataSet without its cell data.
## Added VecFlat class
`vtkm::VecFlat` is a wrapper around a `Vec`-like class that may be a nested
series of vectors. For example, if you run a gradient operation on a vector
field, you are probably going to get a `Vec` of `Vec`s that looks something
like `vtkm::Vec<vtkm::Vec<vtkm::Float32, 3>, 3>`. That is fine, but what if
you want to treat the result simply as a `Vec` of size 9?
The `VecFlat` wrapper class allows you to do this. Simply place the nested
`Vec` as an argument to `VecFlat` and it will behave as a flat `Vec` class.
(In fact, `VecFlat` is a subclass of `Vec`.) The `VecFlat` class can be
copied to and from the nested `Vec` it is wrapping.
There is a `vtkm::make_VecFlat` convenience function that takes an object
and returns a `vtkm::VecFlat` wrapped around it.
`VecFlat` works with any `Vec`-like object as well as scalar values.
However, any type used with `VecFlat` must have `VecTraits` defined and the
number of components must be static (i.e. known at compile time).
## Add a vtkm::[Tuple](Tuple) class
This change added a `vtkm::Tuple` class that is very similar in nature to
`std::tuple`. This should replace our use of tao tuple.
The motivation for this change was some recent attempts at removing objects
like `Invocation` and `FunctionInterface`. I expected these changes to
speed up the build, but in fact they ended up slowing down the build. I
believe the problem was that these required packing variable parameters
into a tuple. I was using the tao `tuple` class, but it seemed to slow down
the compile. (That is, compiling tao's `tuple` seemed much slower than
compiling the equivalent `FunctionInterface` class.)
The implementation of `vtkm::Tuple` is using `pyexpander` to build lots of
simple template cases for the object (with a backup implementation for even
longer argument lists). I believe the compiler is better and parsing
through thousands of lines of simple templates than to employ clever MPL to
build general templates.
### Usage
The `vtkm::Tuple` class is defined in the `vtkm::Tuple.h` header file. A
`Tuple` is designed to behave much like a `std::tuple` with some minor
syntax differences to fit VTK-m coding standards.
A tuple is declared with a list of template argument types.
``` cpp
vtkm::Tuple<vtkm::Id, vtkm::Vec3f, vtkm::cont::ArrayHandle<vtkm::Float32>> myTuple;
If given no arguments, a `vtkm::Tuple` will default-construct its contained
objects. A `vtkm::Tuple` can also be constructed with the initial values of
all contained objects.
``` cpp
vtkm::Tuple<vtkm::Id, vtkm::Vec3f, vtkm::cont::ArrayHandle<vtkm::Float32>>
myTuple(0, vtkm::Vec3f(0, 1, 2), array);
For convenience there is a `vtkm::MakeTuple` function that takes arguments
and packs them into a `Tuple` of the appropriate type. (There is also a
`vtkm::make_tuple` alias to the function to match the `std` version.)
``` cpp
auto myTuple = vtkm::MakeTuple(0, vtkm::Vec3f(0, 1, 2), array);
Data is retrieved from a `Tuple` by using the `vtkm::Get` method. The `Get`
method is templated on the index to get the value from. The index is of
type `vtkm::IdComponent`. (There is also a `vtkm::get` that uses a
`std::size_t` as the index type as an alias to the function to match the
`std` version.)
``` cpp
vtkm::Id a = vtkm::Get<0>(myTuple);
vtkm::Vec3f b = vtkm::Get<1>(myTuple);
vtkm::cont::ArrayHandle<vtkm::Float32> c = vtkm::Get<2>(myTuple);
Likewise `vtkm::TupleSize` and `vtkm::TupleElement` (and their aliases
`vtkm::Tuple_size`, `vtkm::tuple_element`, and `vtkm::tuple_element_t`) are
### Extended Functionality
The `vtkm::Tuple` class contains some functionality beyond that of
`std::tuple` to cover some common use cases in VTK-m that are tricky to
implement. In particular, these methods allow you to use a `Tuple` as you
would commonly use parameter packs. This allows you to stash parameter
packs in a `Tuple` and then get them back out again.
#### For Each
`vtkm::Tuple::ForEach()` is a method that takes a function or functor and
calls it for each of the items in the tuple. Nothing is returned from
`ForEach` and any return value from the function is ignored.
`ForEach` can be used to check the validity of each item.
``` cpp
void CheckPositive(vtkm::Float64 x)
if (x < 0)
throw vtkm::cont::ErrorBadValue("Values need to be positive.");
// ...
vtkm::Tuple<vtkm::Float64, vtkm::Float64, vtkm::Float64> tuple(
CreateValue1(), CreateValue2(), CreateValue3());
// Will throw an error if any of the values are negative.
`ForEach` can also be used to aggregate values.
``` cpp
struct SumFunctor
vtkm::Float64 Sum = 0;
template <typename T>
void operator()(const T& x)
this->Sum = this->Sum + static_cast<vtkm::Float64>(x);
// ...
vtkm::Tuple<vtkm::Float32, vtkm::Float64, vtkm::Id> tuple(
CreateValue1(), CreateValue2(), CreateValue3());
SumFunctor sum;
vtkm::Float64 average = sum.Sum / 3;
#### Transform
`vtkm::Tuple::Transform` is a method that builds a new `Tuple` by calling a
function or functor on each of the items. The return value is placed in the
corresponding part of the resulting `Tuple`, and the type is automatically
created from the return type of the function.
``` cpp
struct GetReadPortalFunctor
template <typename Array>
typename Array::ReadPortal operator()(const Array& array) const
return array.ReadPortal();
// ...
auto arrayTuple = vtkm::MakeTuple(array1, array2, array3);
auto portalTuple = arrayTuple.Transform(GetReadPortalFunctor{});
#### Apply
`vtkm::Tuple::Apply` is a method that calls a function or functor using the
objects in the `Tuple` as the arguments. If the function returns a value,
that value is returned from `Apply`.
``` cpp
struct AddArraysFunctor
template <typename Array1, typename Array2, typename Array3>
vtkm::Id operator()(Array1 inArray1, Array2 inArray2, Array3 outArray) const
vtkm::Id length = inArray1.GetNumberOfValues();
VTKM_ASSERT(inArray2.GetNumberOfValues() == length);
auto inPortal1 = inArray1.ReadPortal();
auto inPortal2 = inArray2.ReadPortal();
auto outPortal = outArray.WritePortal();
for (vtkm::Id index = 0; index < length; ++index)
outPortal.Set(index, inPortal1.Get(index) + inPortal2.Get(index));
return length;
// ...
auto arrayTuple = vtkm::MakeTuple(array1, array2, array3);
vtkm::Id arrayLength = arrayTuple.Apply(AddArraysFunctor{});
If additional arguments are given to `Apply`, they are also passed to the
function (before the objects in the `Tuple`). This is helpful for passing
state to the function. (This feature is not available in either `ForEach`
or `Transform` for technical implementation reasons.)
``` cpp
struct ScanArrayLengthFunctor
template <std::size_t N, typename Array, typename... Remaining>
std::array<vtkm::Id, N + 1 + sizeof...(Remaining)>
operator()(const std::array<vtkm::Id, N>& partialResult,
const Array& nextArray,
const Remaining&... remainingArrays) const
std::array<vtkm::Id, N + 1> nextResult;
std::copy(partialResult.begin(), partialResult.end(), nextResult.begin());
nextResult[N] = nextResult[N - 1] + nextArray.GetNumberOfValues();
return (*this)(nextResult, remainingArray);
template <std::size_t N>
std::array<vtkm::Id, N> operator()(const std::array<vtkm::Id, N>& result) const
return result;
// ...
auto arrayTuple = vtkm::MakeTuple(array1, array2, array3);
std::array<vtkm::Id, 4> =
arrayTuple.Apply(ScanArrayLengthFunctor{}, std::array<vtkm::Id, 1>{ 0 });
## DataSet now only allows unique field names
When you add a `vtkm::cont::Field` to a `vtkm::cont::DataSet`, it now
requires every `Field` to have a unique name. When you attempt to add a
`Field` to a `DataSet` that already has a `Field` of the same name and
association, the old `Field` is removed and replaced with the new `Field`.
You are allowed, however, to have two `Field`s with the same name but
different associations. For example, you could have a point `Field` named
"normals" and also have a cell `Field` named "normals" in the same
This new behavior matches how VTK's data sets manage fields.
The old behavior allowed you to add multiple `Field`s with the same name,
but it would be unclear which one you would get if you asked for a `Field`
by name.
## Result DataSet of coordinate transform has its CoordinateSystem changed
When you run one of the coordinate transform filters,
`CylindricalCoordinateTransform` or `SphericalCoordinateTransform`, the
transform coordiantes are placed as the first `CoordinateSystem` in the
returned `DataSet`. This means that after running this filter, the data
will be moved to this new coordinate space.
Previously, the result of these filters was just placed in a named `Field`
of the output. This caused some confusion because the filter did not seem
to have any effect (unless you knew to modify the output data). Not using
the result as the coordinate system seems like a dubious use case (and not
hard to work around), so this is much better behavior.
## `vtkm::cont::internal::Buffer` now can have ownership transferred
Memory once transferred to `Buffer` always had to be managed by VTK-m. This is problematic
for applications that needed VTK-m to allocate memory, but have the memory ownership
be longer than VTK-m.
`Buffer::TakeHostBufferOwnership` allows for easy transfer ownership of memory out of VTK-m.
When taking ownership of an VTK-m buffer you are provided the following information:
- Memory: A `void*` pointer to the array
- Container: A `void*` pointer used to free the memory. This is necessary to support cases such as allocations transferred into VTK-m from a `std::vector`.
- Delete: The function to call to actually delete the transferred memory
- Reallocate: The function to call to re-allocate the transferred memory. This will throw an exception if users try
to reallocate a buffer that was 'view' only
- Size: The size in number of elements of the array
To properly steal memory from VTK-m you do the following:
vtkm::cont::ArrayHandle<T> arrayHandle;
auto stolen = arrayHandle.GetBuffers()->TakeHostBufferOwnership();
## Configurable default types
Because VTK-m compiles efficient code for accelerator architectures, it
often has to compile for static types. This means that dynamic types often
have to be determined at runtime and converted to static types. This is the
reason for the `CastAndCall` architecture in VTK-m.
For this `CastAndCall` to work, there has to be a finite set of static
types to try at runtime. If you don't compile in the types you need, you
will get runtime errors. However, the more types you compile in, the longer
the compile time and executable size. Thus, getting the types right is
The "right" types to use can change depending on the application using
VTK-m. For example, when VTK links in VTK-m, it needs to support lots of
types and can sacrifice the compile times to do so. However, if using VTK-m
in situ with a fortran simulation, space and time are critical and you
might only need to worry about double SoA arrays.
Thus, it is important to customize what types VTK-m uses based on the
application. This leads to the oxymoronic phrase of configuring the default
types used by VTK-m.
This is being implemented by providing VTK-m with a header file that
defines the default types. The header file provided to VTK-m should define
one or more of the following preprocessor macros:
* `VTKM_DEFAULT_TYPE_LIST` - a `vtkm::List` of value types for fields that
filters should directly operate on (where applicable).
* `VTKM_DEFAULT_STORAGE_LIST` - a `vtkm::List` of storage tags for fields
that filters should directly operate on.
`vtkm::cont::CellSet` types that filters should operate on as a
strutured cell set.
`vtkm::cont::CellSet` types that filters should operate on as an
unstrutured cell set.
* `VTKM_DEFAULT_CELL_SET_LIST` - a `vtkm::List` of `vtkm::cont::CellSet`
types that filters should operate on (where applicable). The default of
is usually correct.
If any of these macros are not defined, a default version will be defined.
(This is the same default used if no header file is provided.)
This header file is provided to the build by setting the
points to the file, which will be configured and copied to VTK-m's build
For convenience, header files can be added to the VTK_m source directory