release-notes.md 112 KB
Newer Older
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
VTK-m 1.6 Release Notes
=======================

# Table of Contents

1. [Core](#Core)
  - Add Kokkos backend
  - Deprecate `DataSetFieldAdd`
  - Move VTK file readers and writers into vtkm_io
  - Remove VTKDataSetWriter::WriteDataSet just_points parameter
  - Added VecFlat class
  - Add a vtkm::Tuple class
  - DataSet now only allows unique field names
  - Result DataSet of coordinate transform has its CoordinateSystem changed
  - `vtkm::cont::internal::Buffer` now can have ownership transferred
  - Configurable default types
2. [ArrayHandle](#ArrayHandle)
  - Shorter fancy array handle classnames
  - `ReadPortal().Get(idx)`
  - Precompiled `ArrayCopy` for `UnknownArrayHandle`
  - Create `ArrayHandleOffsetsToNumComponents`
  - Recombine extracted component arrays from unknown arrays
  - UnknownArrayHandle and UncertainArrayHandle for runtime-determined types
  - Support `ArrayHandleSOA` as a "default" array
  - Removed old `ArrayHandle` transfer mechanism
  - Order asynchronous `ArrayHandle` access
  - Improvements to moving data into ArrayHandle
  - Deprecate ArrayHandleVirtualCoordinates
  - ArrayHandleDecorator Allocate and Shrink Support
  - Portals may advertise custom iterators
  - Redesign of ArrayHandle to access data using typeless buffers
  - `ArrayRangeCompute` works on any array type without compiling device code
  - Implemented ArrayHandleRandomUniformBits and ArrayHandleRandomUniformReal
  - Extract component arrays from unknown arrays
  - `ArrayHandleGroupVecVariable` holds now one more offset.
3. [Control Environment](#Control-Environment)
  - Algorithms for Control and Execution Environments
4. [Execution Environment](#Execution-Environment)
  - Scope ExecObjects with Tokens
  - Masks and Scatters Supported for 3D Scheduling
  - Virtual methods in execution environment deprecated
  - Deprecate Execute with policy
5. [Worklets and Filters](#Worklets-and-Filters)
  - Enable setting invalid value in probe filter
  - Avoid raising errors when operating on cells
  - Add atomic free functions
  - Flying Edges
  - Filters specify their own field types
6. [Build](#Build)
  - Disable asserts for CUDA architecture builds
  - Disable asserts for HIP architecture builds
  - Add VTKM_DEPRECATED macro
7. [Other](#Other)
  - Porting layer for future std features
  - Removed OpenGL Rendering Classes
  - Reorganization of `io` directory
  - Implemented PNG/PPM image Readers/Writers
  - Updated Benchmark Framework
  - Provide scripts to build Gitlab-ci workers locally
  - Replaced `vtkm::ListTag` with `vtkm::List`
  - Add `ListTagRemoveIf`
  - Write uniform and rectilinear grids to legacy VTK files
63
8. [References](#References)
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136

# Core

## Add Kokkos backend

  Adds a new device backend `Kokkos` which uses the kokkos library for parallelism.
  User must provide the kokkos build and Vtk-m will use the default configured execution
  space.

## Deprecate `DataSetFieldAdd`

The class `vtkm::cont::DataSetFieldAdd` is now deprecated.
Its methods, `AddPointField` and `AddCellField` have been moved to member functions
of `vtkm::cont::DataSet`, which simplifies many calls.

For example, the following code

```cpp
vtkm::cont::DataSetFieldAdd fieldAdder;
fieldAdder.AddCellField(dataSet, "cellvar", values);
```

would now be

```cpp
dataSet.AddCellField("cellvar", values);
```

## Move VTK file readers and writers into vtkm_io

The legacy VTK file reader and writer were created back when VTK-m was a
header-only library. Things have changed and we now compile quite a bit of
code into libraries. At this point, there is no reason why the VTK file
reader/writer should be any different.

Thus, `VTKDataSetReader`, `VTKDataSetWriter`, and several supporting
classes are now compiled into the `vtkm_io` library. Also similarly updated
`BOVDataSetReader` for good measure.

As a side effect, code using VTK-m will need to link to `vtkm_io` if they
are using any readers or writers.


## Remove VTKDataSetWriter::WriteDataSet just_points parameter

In the method `VTKDataSetWriter::WriteDataSet`, `just_points` parameter has been
removed due to lack of usage. 

The purpose of `just_points` was to allow exporting only the points of a
DataSet without its cell data.


## Added VecFlat class

`vtkm::VecFlat` is a wrapper around a `Vec`-like class that may be a nested
series of vectors. For example, if you run a gradient operation on a vector
field, you are probably going to get a `Vec` of `Vec`s that looks something
like `vtkm::Vec<vtkm::Vec<vtkm::Float32, 3>, 3>`. That is fine, but what if
you want to treat the result simply as a `Vec` of size 9?

The `VecFlat` wrapper class allows you to do this. Simply place the nested
`Vec` as an argument to `VecFlat` and it will behave as a flat `Vec` class.
(In fact, `VecFlat` is a subclass of `Vec`.) The `VecFlat` class can be
copied to and from the nested `Vec` it is wrapping.

There is a `vtkm::make_VecFlat` convenience function that takes an object
and returns a `vtkm::VecFlat` wrapped around it.

`VecFlat` works with any `Vec`-like object as well as scalar values.
However, any type used with `VecFlat` must have `VecTraits` defined and the
number of components must be static (i.e. known at compile time).


137
## Add a vtkm::[Tuple](Tuple) class
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155

This change added a `vtkm::Tuple` class that is very similar in nature to
`std::tuple`. This should replace our use of tao tuple.

The motivation for this change was some recent attempts at removing objects
like `Invocation` and `FunctionInterface`. I expected these changes to
speed up the build, but in fact they ended up slowing down the build. I
believe the problem was that these required packing variable parameters
into a tuple. I was using the tao `tuple` class, but it seemed to slow down
the compile. (That is, compiling tao's `tuple` seemed much slower than
compiling the equivalent `FunctionInterface` class.)

The implementation of `vtkm::Tuple` is using `pyexpander` to build lots of
simple template cases for the object (with a backup implementation for even
longer argument lists). I believe the compiler is better and parsing
through thousands of lines of simple templates than to employ clever MPL to
build general templates.

156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
### Usage

The `vtkm::Tuple` class is defined in the `vtkm::Tuple.h` header file. A
`Tuple` is designed to behave much like a `std::tuple` with some minor
syntax differences to fit VTK-m coding standards.

A tuple is declared with a list of template argument types.

``` cpp
vtkm::Tuple<vtkm::Id, vtkm::Vec3f, vtkm::cont::ArrayHandle<vtkm::Float32>> myTuple;
```

If given no arguments, a `vtkm::Tuple` will default-construct its contained
objects. A `vtkm::Tuple` can also be constructed with the initial values of
all contained objects.

``` cpp
vtkm::Tuple<vtkm::Id, vtkm::Vec3f, vtkm::cont::ArrayHandle<vtkm::Float32>> 
  myTuple(0, vtkm::Vec3f(0, 1, 2), array);
```

For convenience there is a `vtkm::MakeTuple` function that takes arguments
and packs them into a `Tuple` of the appropriate type. (There is also a
`vtkm::make_tuple` alias to the function to match the `std` version.)

``` cpp
auto myTuple = vtkm::MakeTuple(0, vtkm::Vec3f(0, 1, 2), array);
```

Data is retrieved from a `Tuple` by using the `vtkm::Get` method. The `Get`
method is templated on the index to get the value from. The index is of
type `vtkm::IdComponent`. (There is also a `vtkm::get` that uses a
`std::size_t` as the index type as an alias to the function to match the
`std` version.)

``` cpp
vtkm::Id a = vtkm::Get<0>(myTuple);
vtkm::Vec3f b = vtkm::Get<1>(myTuple);
vtkm::cont::ArrayHandle<vtkm::Float32> c = vtkm::Get<2>(myTuple);
```

Likewise `vtkm::TupleSize` and `vtkm::TupleElement` (and their aliases
`vtkm::Tuple_size`, `vtkm::tuple_element`, and `vtkm::tuple_element_t`) are
provided.

### Extended Functionality

The `vtkm::Tuple` class contains some functionality beyond that of
`std::tuple` to cover some common use cases in VTK-m that are tricky to
implement. In particular, these methods allow you to use a `Tuple` as you
would commonly use parameter packs. This allows you to stash parameter
packs in a `Tuple` and then get them back out again.

#### For Each

`vtkm::Tuple::ForEach()` is a method that takes a function or functor and
calls it for each of the items in the tuple. Nothing is returned from
`ForEach` and any return value from the function is ignored.

`ForEach` can be used to check the validity of each item.

``` cpp
void CheckPositive(vtkm::Float64 x)
{
  if (x < 0)
  {
    throw vtkm::cont::ErrorBadValue("Values need to be positive.");
  }
}

// ...

  vtkm::Tuple<vtkm::Float64, vtkm::Float64, vtkm::Float64> tuple(
    CreateValue1(), CreateValue2(), CreateValue3());

  // Will throw an error if any of the values are negative.
  tuple.ForEach(CheckPositive);
```

`ForEach` can also be used to aggregate values.

``` cpp
struct SumFunctor
{
  vtkm::Float64 Sum = 0;
  
  template <typename T>
  void operator()(const T& x)
  {
    this->Sum = this->Sum + static_cast<vtkm::Float64>(x);
  }
};

// ...

  vtkm::Tuple<vtkm::Float32, vtkm::Float64, vtkm::Id> tuple(
    CreateValue1(), CreateValue2(), CreateValue3());

  SumFunctor sum;
  tuple.ForEach(sum);
  vtkm::Float64 average = sum.Sum / 3;
```

#### Transform

`vtkm::Tuple::Transform` is a method that builds a new `Tuple` by calling a
function or functor on each of the items. The return value is placed in the
corresponding part of the resulting `Tuple`, and the type is automatically
created from the return type of the function.

``` cpp
struct GetReadPortalFunctor
{
  template <typename Array>
  typename Array::ReadPortal operator()(const Array& array) const
  {
    VTKM_IS_ARRAY_HANDLE(Array);
	return array.ReadPortal();
  }
};

// ...

  auto arrayTuple = vtkm::MakeTuple(array1, array2, array3);
  
  auto portalTuple = arrayTuple.Transform(GetReadPortalFunctor{});
```

#### Apply

`vtkm::Tuple::Apply` is a method that calls a function or functor using the
objects in the `Tuple` as the arguments. If the function returns a value,
that value is returned from `Apply`.

``` cpp
struct AddArraysFunctor
{
  template <typename Array1, typename Array2, typename Array3>
  vtkm::Id operator()(Array1 inArray1, Array2 inArray2, Array3 outArray) const
  {
    VTKM_IS_ARRAY_HANDLE(Array1);
    VTKM_IS_ARRAY_HANDLE(Array2);
    VTKM_IS_ARRAY_HANDLE(Array3);

    vtkm::Id length = inArray1.GetNumberOfValues();
	VTKM_ASSERT(inArray2.GetNumberOfValues() == length);
	outArray.Allocate(length);
	
	auto inPortal1 = inArray1.ReadPortal();
	auto inPortal2 = inArray2.ReadPortal();
	auto outPortal = outArray.WritePortal();
	for (vtkm::Id index = 0; index < length; ++index)
	{
	  outPortal.Set(index, inPortal1.Get(index) + inPortal2.Get(index));
	}
	
	return length;
  }
};

// ...

  auto arrayTuple = vtkm::MakeTuple(array1, array2, array3);

  vtkm::Id arrayLength = arrayTuple.Apply(AddArraysFunctor{});
```

If additional arguments are given to `Apply`, they are also passed to the
function (before the objects in the `Tuple`). This is helpful for passing
state to the function. (This feature is not available in either `ForEach`
or `Transform` for technical implementation reasons.)

``` cpp
struct ScanArrayLengthFunctor
{
  template <std::size_t N, typename Array, typename... Remaining>
  std::array<vtkm::Id, N + 1 + sizeof...(Remaining)>
  operator()(const std::array<vtkm::Id, N>& partialResult,
             const Array& nextArray,
			 const Remaining&... remainingArrays) const
  {
    std::array<vtkm::Id, N + 1> nextResult;
	std::copy(partialResult.begin(), partialResult.end(), nextResult.begin());
    nextResult[N] = nextResult[N - 1] + nextArray.GetNumberOfValues();
	return (*this)(nextResult, remainingArray);
  }
  
  template <std::size_t N>
  std::array<vtkm::Id, N> operator()(const std::array<vtkm::Id, N>& result) const
  {
    return result;
  }
};

// ...

  auto arrayTuple = vtkm::MakeTuple(array1, array2, array3);
  
  std::array<vtkm::Id, 4> = 
    arrayTuple.Apply(ScanArrayLengthFunctor{}, std::array<vtkm::Id, 1>{ 0 });
```

358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
## DataSet now only allows unique field names

When you add a `vtkm::cont::Field` to a `vtkm::cont::DataSet`, it now
requires every `Field` to have a unique name. When you attempt to add a
`Field` to a `DataSet` that already has a `Field` of the same name and
association, the old `Field` is removed and replaced with the new `Field`.

You are allowed, however, to have two `Field`s with the same name but
different associations. For example, you could have a point `Field` named
"normals" and also have a cell `Field` named "normals" in the same
`DataSet`.

This new behavior matches how VTK's data sets manage fields.

The old behavior allowed you to add multiple `Field`s with the same name,
but it would be unclear which one you would get if you asked for a `Field`
by name.


## Result DataSet of coordinate transform has its CoordinateSystem changed

When you run one of the coordinate transform filters,
`CylindricalCoordinateTransform` or `SphericalCoordinateTransform`, the
transform coordiantes are placed as the first `CoordinateSystem` in the
returned `DataSet`. This means that after running this filter, the data
will be moved to this new coordinate space.

Previously, the result of these filters was just placed in a named `Field`
of the output. This caused some confusion because the filter did not seem
to have any effect (unless you knew to modify the output data). Not using
the result as the coordinate system seems like a dubious use case (and not
hard to work around), so this is much better behavior.


## `vtkm::cont::internal::Buffer` now can have ownership transferred

Memory once transferred to `Buffer` always had to be managed by VTK-m. This is problematic
for applications that needed VTK-m to allocate memory, but have the memory ownership
be longer than VTK-m.

`Buffer::TakeHostBufferOwnership` allows for easy transfer ownership of memory out of VTK-m.
When taking ownership of an VTK-m buffer you are provided the following information:

- Memory: A `void*` pointer to the array
- Container: A `void*` pointer used to free the memory. This is necessary to support cases such as allocations transferred into VTK-m from a `std::vector`.
- Delete: The function to call to actually delete the transferred memory
- Reallocate: The function to call to re-allocate the transferred memory. This will throw an exception if users try
to reallocate a buffer that was 'view' only
- Size: The size in number of elements of the array 

 
To properly steal memory from VTK-m you do the following:
```cpp
  vtkm::cont::ArrayHandle<T> arrayHandle;

  ...

  auto stolen = arrayHandle.GetBuffers()->TakeHostBufferOwnership();
    
  ...

  stolen.Delete(stolen.Container);
```


## Configurable default types

Because VTK-m compiles efficient code for accelerator architectures, it
often has to compile for static types. This means that dynamic types often
have to be determined at runtime and converted to static types. This is the
reason for the `CastAndCall` architecture in VTK-m.

For this `CastAndCall` to work, there has to be a finite set of static
types to try at runtime. If you don't compile in the types you need, you
will get runtime errors. However, the more types you compile in, the longer
the compile time and executable size. Thus, getting the types right is
important.

The "right" types to use can change depending on the application using
VTK-m. For example, when VTK links in VTK-m, it needs to support lots of
types and can sacrifice the compile times to do so. However, if using VTK-m
in situ with a fortran simulation, space and time are critical and you
might only need to worry about double SoA arrays.

Thus, it is important to customize what types VTK-m uses based on the
application. This leads to the oxymoronic phrase of configuring the default
types used by VTK-m.

This is being implemented by providing VTK-m with a header file that
defines the default types. The header file provided to VTK-m should define
one or more of the following preprocessor macros:

  * `VTKM_DEFAULT_TYPE_LIST` - a `vtkm::List` of value types for fields that
     filters should directly operate on (where applicable).
  * `VTKM_DEFAULT_STORAGE_LIST` - a `vtkm::List` of storage tags for fields
     that filters should directly operate on.
  * `VTKM_DEFAULT_CELL_SET_LIST_STRUCTURED` - a `vtkm::List` of
     `vtkm::cont::CellSet` types that filters should operate on as a
     strutured cell set.
  * `VTKM_DEFAULT_CELL_SET_LIST_UNSTRUCTURED` - a `vtkm::List` of
     `vtkm::cont::CellSet` types that filters should operate on as an
     unstrutured cell set.
  * `VTKM_DEFAULT_CELL_SET_LIST` - a `vtkm::List` of `vtkm::cont::CellSet`
     types that filters should operate on (where applicable). The default of
     `vtkm::ListAppend<VTKM_DEFAULT_CELL_SET_LIST_STRUCTURED, VTKM_DEFAULT_CELL_SET_LIST>`
	 is usually correct.

If any of these macros are not defined, a default version will be defined.
(This is the same default used if no header file is provided.)

This header file is provided to the build by setting the
`VTKm_DEFAULT_TYPES_HEADER` CMake variable. `VTKm_DEFAULT_TYPES_HEADER`
points to the file, which will be configured and copied to VTK-m's build
directory.

For convenience, header files can be added to the VTK_m source directory
(conventionally under vtkm/cont/internal). If this is the case, an advanced
CMake option should be added to select the provided header file.


478
479

# ArrayHandle
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501

## Shorter fancy array handle classnames

Many of the fancy `ArrayHandle`s use the generic builders like
`ArrayHandleTransform` and `ArrayHandleImplicit` for their implementation.
Such is fine, but because they use functors and other such generic items to
template their `Storage`, you can end up with very verbose classnames. This
is an issue for humans trying to discern classnames. It can also be an
issue for compilers that end up with very long resolved classnames that
might get truncated if they extend past what was expected.

The fix was for these classes to declare their own `Storage` tag and then
implement their `Storage` and `ArrayTransport` classes as trivial
subclasses of the generic `ArrayHandleImplicit` or `ArrayHandleTransport`.

As an added bonus, a lot of this shortening also means that storage that
relies on other array handles now are just typed by the storage of the
decorated type, not the array itself. This should make the types a little
more robust.

Here is a list of classes that were updated.

502
### `ArrayHandleCast<TargetT, vtkm::cont::ArrayHandle<SourceT, SourceStorage>>`
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518

Old storage: 
``` cpp
vtkm::cont::internal::StorageTagTransform<
  vtkm::cont::ArrayHandle<SourceT, SourceStorage>,
  vtkm::cont::internal::Cast<TargetT, SourceT>,
  vtkm::cont::internal::Cast<SourceT, TargetT>>
```

New Storage:
``` cpp
vtkm::cont::StorageTagCast<SourceT, SourceStorage>
```

(Developer's note: Implementing this change to `ArrayHandleCast` was a much bigger PITA than expected.)

519
### `ArrayHandleCartesianProduct<AH1, AH2, AH3>`
520
521
522
523
524
525
526
527
528
529
530
531
532
533

Old storage:
``` cpp
vtkm::cont::internal::StorageTagCartesianProduct<
  vtkm::cont::ArrayHandle<ValueType, StorageTag1,
  vtkm::cont::ArrayHandle<ValueType, StorageTag2,
  vtkm::cont::ArrayHandle<ValueType, StorageTag3>>
```

New storage:
``` cpp
vtkm::cont::StorageTagCartesianProduct<StorageTag1, StorageTag2, StorageTag3>
```

534
### `ArrayHandleCompositeVector<AH1, AH2, ...>`
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551

Old storage:
``` cpp
vtkm::cont::internal::StorageTagCompositeVector<
  tao::tuple<
    vtkm::cont::ArrayHandle<ValueType, StorageType1>, 
	vtkm::cont::ArrayHandle<ValueType, StorageType2>,
	...
  >
>
```

New storage:
``` cpp
vtkm::cont::StorageTagCompositeVec<StorageType1, StorageType2>
```

552
### `ArrayHandleConcatinate`
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584

First an example with two simple types.

Old storage:
``` cpp
vtkm::cont::StorageTagConcatenate<
  vtkm::cont::ArrayHandle<ValueType, StorageTag1>,
  vtkm::cont::ArrayHandle<ValueType, StorageTag2>>
```

New storage:
``` cpp
vtkm::cont::StorageTagConcatenate<StorageTag1, StorageTag2>
```

Now a more specific example taken from the unit test of a concatination of a concatination.

Old storage:
``` cpp
vtkm::cont::StorageTagConcatenate<
  vtkm::cont::ArrayHandleConcatenate<
    vtkm::cont::ArrayHandle<ValueType, StorageTag1>,
	vtkm::cont::ArrayHandle<ValueType, StorageTag2>>,
  vtkm::cont::ArrayHandle<ValueType, StorageTag3>>
```

New storage:
``` cpp
vtkm::cont::StorageTagConcatenate<
  vtkm::cont::StorageTagConcatenate<StorageTag1, StorageTag2>, StorageTag3>
```

585
### `ArrayHandleConstant`
586
587
588
589
590
591
592
593
594
595
596
597
598

Old storage:
``` cpp
vtkm::cont::StorageTagImplicit<
  vtkm::cont::detail::ArrayPortalImplicit<
    vtkm::cont::detail::ConstantFunctor<ValueType>>>
```

New storage:
``` cpp
vtkm::cont::StorageTagConstant
```

599
### `ArrayHandleCounting`
600
601
602
603
604
605
606
607
608
609
610

Old storage:
``` cpp
vtkm::cont::StorageTagImplicit<vtkm::cont::internal::ArrayPortalCounting<ValueType>>
```

New storage:
``` cpp
vtkm::cont::StorageTagCounting
```

611
### `ArrayHandleGroupVec`
612
613
614
615
616
617
618
619
620
621
622
623

Old storage:
``` cpp
vtkm::cont::internal::StorageTagGroupVec<
  vtkm::cont::ArrayHandle<ValueType, StorageTag>, N>
```

New storage:
``` cpp
vtkm::cont::StorageTagGroupVec<StorageTag, N>
```

624
### `ArrayHandleGroupVecVariable`
625
626
627
628
629
630
631
632
633
634
635
636
637

Old storage:
``` cpp
vtkm::cont::internal::StorageTagGroupVecVariable<
  vtkm::cont::ArrayHandle<ValueType, StorageTag1>, 
  vtkm::cont::ArrayHandle<vtkm::Id, StorageTag2>>
```

New storage:
``` cpp
vtkm::cont::StorageTagGroupVecVariable<StorageTag1, StorageTag2>
```

638
### `ArrayHandleIndex`
639
640
641
642
643
644
645
646
647
648
649
650

Old storage:
``` cpp
vtkm::cont::StorageTagImplicit<
  vtkm::cont::detail::ArrayPortalImplicit<vtkm::cont::detail::IndexFunctor>>
```

New storage:
``` cpp
vtkm::cont::StorageTagIndex
```

651
### `ArrayHandlePermutation`
652
653
654
655
656
657
658
659
660
661
662
663
664

Old storage:
``` cpp
vtkm::cont::internal::StorageTagPermutation<
  vtkm::cont::ArrayHandle<vtkm::Id, StorageTag1>,
  vtkm::cont::ArrayHandle<ValueType, StorageTag2>>
```

New storage:
``` cpp
vtkm::cont::StorageTagPermutation<StorageTag1, StorageTag2>
```

665
### `ArrayHandleReverse`
666
667
668
669
670
671
672
673
674
675
676

Old storage:
``` cpp
vtkm::cont::StorageTagReverse<vtkm::cont::ArrayHandle<ValueType, vtkm::cont::StorageTag>>
```

New storage:
``` cpp
vtkm::cont::StorageTagReverse<StorageTag>
```

677
### `ArrayHandleUniformPointCoordinates`
678
679
680
681
682
683
684
685
686
687
688

Old storage:
``` cpp
vtkm::cont::StorageTagImplicit<vtkm::internal::ArrayPortalUniformPointCoordinates>
```

New Storage:
``` cpp
vtkm::cont::StorageTagUniformPoints
```

689
### `ArrayHandleView`
690
691
692
693
694
695
696
697
698
699
700
701

Old storage:
``` cpp
vtkm::cont::StorageTagView<vtkm::cont::ArrayHandle<ValueType, StorageTag>>
```

New storage:
``` cpp
'vtkm::cont::StorageTagView<StorageTag>
```


702
### `ArrayPortalZip`
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814

Old storage:
``` cpp
vtkm::cont::internal::StorageTagZip<
  vtkm::cont::ArrayHandle<ValueType1, StorageTag1>,
  vtkm::cont::ArrayHandle<ValueType2, StorageTag2>>
```

New storage:
``` cpp
vtkm::cont::StorageTagZip<StorageTag1, StorageTag2>
```


## `ReadPortal().Get(idx)`

Calling `ReadPortal()` in a tight loop is an antipattern.
A call to `ReadPortal()` causes the array to be copied back to the control environment,
and hence code like

```cpp
for (vtkm::Id i = 0; i < array.GetNumberOfValues(); ++i) {
    vtkm::FloatDefault x = array.ReadPortal().Get(i);
}
```

is a quadratic-scaling loop.

We have remove *almost* all internal uses of the `ReadPortal().Get` antipattern,
with the exception of 4 API calls into which the pattern is baked in:
`CellSetExplicit::GetCellShape`, `CellSetPermutation::GetNumberOfPointsInCell`, `CellSetPermutation::GetCellShape`, and `CellSetPermutation::GetCellPointIds`.
We expect these will need to be deprecated in the future.

## Precompiled `ArrayCopy` for `UnknownArrayHandle`

Previously, in order to copy an `UnknownArrayHandle`, you had to specify
some subset of types and then specially compile a copy for each potential
type. With the new ability to extract a component from an
`UnknownArrayHandle`, it is now feasible to precompile copying an
`UnknownArrayHandle` to another array. This greatly reduces the overhead of
using `ArrayCopy` to copy `UnknownArrayHandle`s while simultaneously
increasing the likelihood that the copy will be successful.

## Create `ArrayHandleOffsetsToNumComponents`

`ArrayHandleOffsetsToNumComponents` is a fancy array that takes an array of
offsets and converts it to an array of the number of components for each
packed entry.

It is common in VTK-m to pack small vectors of variable sizes into a single
contiguous array. For example, cells in an explicit cell set can each have
a different amount of vertices (triangles = 3, quads = 4, tetra = 4, hexa =
8, etc.). Generally, to access items in this list, you need an array of
components in each entry and the offset for each entry. However, if you
have just the array of offsets in sorted order, you can easily derive the
number of components for each entry by subtracting adjacent entries. This
works best if the offsets array has a size that is one more than the number
of packed vectors with the first entry set to 0 and the last entry set to
the total size of the packed array (the offset to the end).

When packing data of this nature, it is common to start with an array that
is the number of components. You can convert that to an offsets array using
the `vtkm::cont::ConvertNumComponentsToOffsets` function. This will create
an offsets array with one extra entry as previously described. You can then
throw out the original number of components array and use the offsets with
`ArrayHandleOffsetsToNumComponents` to represent both the offsets and num
components while storing only one array.

This replaces the use of `ArrayHandleDecorator` in `CellSetExplicit`.
The two implementation should do the same thing, but the new
`ArrayHandleOffsetsToNumComponents` should be less complex for
compilers.

## Recombine extracted component arrays from unknown arrays

Building on the recent capability to [extract component arrays from unknown
arrays](array-extract-component.md), there is now also the ability to
recombine these extracted arrays to a single `ArrayHandle`. It might seem
counterintuitive to break an `ArrayHandle` into component arrays and then
combine the component arrays back into a single `ArrayHandle`, but this is
a very handy way to run algorithms without knowing the exact `ArrayHandle`
type.

Recall that when extracting a component array from an `UnknownArrayHandle`
you only need to know the base component of the value type of the contained
`ArrayHandle`. That makes extracting a component array independent from
either the size of any `Vec` value type and any storage type.

The added `UnknownArrayHandle::ExtractArrayFromComponents` method allows
you to use the functionality to transform the unknown array handle to a
form of `ArrayHandle` that depends only on this base component type. This
method internally uses a new `ArrayHandleRecombineVec` class, but this
class is mostly intended for internal use by this class.

As an added convenience, `UnknownArrayHandle` now also provides the
`CastAndCallWithExtractedArray` method. This method works like other
`CastAndCall`s except that it uses the `ExtractArrayFromComponents` feature
to allow you to handle most `ArrayHandle` types with few template
instances.


## UnknownArrayHandle and UncertainArrayHandle for runtime-determined types

Two new classes have been added to VTK-m: `UnknownArrayHandle` and
`UncertainArrayHandle`. These classes serve the same purpose as the set of
`VariantArrayHandle` classes and will replace them.

Motivated mostly by the desire to move away from `ArrayHandleVirtual`, we
have multiple reasons to completely refactor the `VariantArrayHandle`
class. These include changing the implementation, some behavior, and even
the name.

815
### Motivation
816
817
818
819

We have several reasons that have accumulated to revisit the implementation
of `VariantArrayHandle`.

820
#### Move away from `ArrayHandleVirtual`
821
822
823
824
825
826
827
828
829
830
831
832
833

The current implementation of `VariantArrayHandle` internally stores the
array wrapped in an `ArrayHandleVirtual`. That makes sense since you might
as well consolidate the hierarchy of virtual objects into one.

Except `ArrayHandleVirtual` is being deprecated, so it no longer makes
sense to use that internally.

So we will transition the class back to managing the data as typeless on
its own. We will consider using function pointers rather than actual
virtual functions because compilers can be slow in creating lots of virtual
subclasses.

834
#### Reintroduce storage tag lists
835
836
837
838
839
840
841
842
843
844
845
846

The original implementation of `VariantArrayHandle` (which at the time was
called `DynamicArrayHandle`) actually had two type lists: one for the array
value type and one for the storage type. The storage type list was removed
soon after `ArrayHandleVirtual` was introduced because whatever the type of
array it could be access as `ArrayHandleVirtual`.

However, with `ArrayHandleVirtual` being deprecated, this feature is no
longer possible. We are in need again for the list of storage types to try.
Thus, we need to reintroduce this template argument to
`VariantArrayHandle`.

847
#### More clear name
848
849
850
851
852
853
854
855
856

The name of this class has always been unsatisfactory. The first name,
`DynamicArrayHandle`, makes it sound like the data is always changing. The
second name, `VariantArrayHandle`, makes it sound like an array that holds
a value type that can vary (like an `std::variant`).

We can use a more clear name that expresses better that it is holding an
`ArrayHandle` of an _unknown_ type.

857
#### Take advantage of default types for less templating
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876

Once upon a time everything in VTK-m was templated header library. Things
have changed quite a bit since then. The most recent development is the
ability to select the "default types" with CMake configuration that allows
you to select a global set of types you care about during compilation. This
is so units like filters can be compiled into a library with all types we
care about, and we don't have to constantly recompile units.

This means that we are becoming less concerned about maintaining type lists
everywhere. Often we can drop the type list and pass data across libraries.

With that in mind, it makes less sense for `VariantArrayHandle` to actually
be a `using` alias for `VariantArrayHandleBase<VTKM_DEFAULT_TYPE_LIST>`.

In response, we can revert the is-a relationship between the two. Have a
completely typeless version as the base class and have a second version
templated version to express when the type of the array has been partially
narrowed down to given type lists.

877
### New Name and Structure
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902

The ultimate purpose of this class is to store an `ArrayHandle` where the
value and storage types are unknown. Thus, an appropriate name for the
class is `UnknownArrayHandle`.

`UnknownArrayHandle` is _not_ templated. It simply stores an `ArrayHandle`
in a typeless (`void *`) buffer. It does, however, contain many templated
methods that allow you to query whether the contained array matches given
types, to cast to given types, and to cast and call to a given functor
(from either given type lists or default lists).

Rather than have a virtual class structure to manage the typeless array,
the new management will use function pointers. This has shown to sometimes
improve compile times and generate less code.

Sometimes it is the case that the set of potential types can be narrowed. In
this case, the array ceases to be unknown and becomes _uncertain_. Thus,
the companion class to `UnknownArrayHandle` is `UncertainArrayHandle`.

`UncertainArrayHandle` has two template parameters: a list of potential
value types and a list of potential storage types. The behavior of
`UncertainArrayHandle` matches that of `UnknownArrayHandle` (and might
inherit from it). However, for `CastAndCall` operations, it will use the
type lists defined in its template parameters.

903
### Serializing UnknownArrayHandle
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972

Because `UnknownArrayHandle` is not templated, it contains some
opportunities to compile things into the `vtkm_cont` library. Templated
methods like `CastAndCall` cannot be, but the specializations of DIY's
serialize can be.

And since it only has to be compiled once into a library, we can spend some
extra time compiling for more types. We don't have to restrict ourselves to
`VTKM_DEFAULT_TYPE_LIST`. We can compile for vtkm::TypeListTagAll.


Support `ArrayHandleSOA` as a "default" array

Many programs, particularly simulations, store fields of vectors in
separate arrays for each component. This maps to the storage of
`ArrayHandleSOA`. The VTK-m code tends to prefer the AOS storage (which is
what is implemented in `ArrayHandleBasic`, and the behavior of which is
inherited from VTK). VTK-m should better support adding `ArrayHandleSOA` as
one of the types.

We now have a set of default types for Ascent that uses SOA as one of the
basic types.

Part of this change includes an intentional feature regression of
`ArrayHandleSOA` to only support value types of `Vec`. Previously, scalar
types were supported. However, the behavior of `ArrayHandleSOA` is exactly
the same as `ArrayHandleBasic`, except a lot more template code has to be
generated. That itself is not a huge deal, but because you have 2 types
that essentially do the same thing, a lot of template code in VTK-m would
unwind to create two separate code paths that do the same thing with the
same data. To avoid creating those code paths, we simply make any use of
`ArrayHandleSOA` without a `Vec` value invalid. This will prevent VTK-m
from creating those code paths.


## Removed old `ArrayHandle` transfer mechanism

Deleted the default implementation of `ArrayTransfer`. `ArrayTransfer` is
used with the old `ArrayHandle` style to move data between host and device.
The new version of `ArrayHandle` does not use `ArrayTransfer` at all
because this functionality is wrapped in `Buffer` (where it can exist in a
precompiled library).

Once all the old `ArrayHandle` classes are gone, this class will be removed
completely. Although all the remaining `ArrayHandle` classes provide their
own versions of `ArrayTransfer`, they still need the prototype to be
defined to specialize. Thus, the guts of the default `ArrayTransfer` are
removed and replaced with a compile error if you try to compile it.

Also removed `ArrayManagerExecution`. This class was used indirectly by the
old `ArrayHandle`, through `ArrayHandleTransfer`, to move data to and from
a device. This functionality has been replaced in the new `ArrayHandle`s
through the `Buffer` class (which can be compiled into libraries rather
than make every translation unit compile their own template).


## Order asynchronous `ArrayHandle` access

The recent feature of [tokens that scope access to
`ArrayHandle`s](scoping-tokens.md) allows multiple threads to use the same
`ArrayHandle`s without read/write hazards. The intent is twofold. First, it
allows two separate threads in the control environment to independently
schedule tasks. Second, it allows us to move toward scheduling worklets and
other algorithms asynchronously.

However, there was a flaw with the original implementation. Once requests
to an `ArrayHandle` get queued up, they are resolved in arbitrary order.
This might mean that things run in surprising and incorrect order.

973
### Problematic use case
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000

To demonstrate the flaw in the original implementation, let us consider a
future scenario where when you invoke a worklet (on OpenMP or TBB), the
call to invoke returns immediately and the actual work is scheduled
asynchronously. Now let us say we have a sequence of 3 worklets we wish to
run: `Worklet1`, `Worklet2`, and `Worklet3`. One of `Worklet1`'s parameters
is a `FieldOut` that creates an intermediate `ArrayHandle` that we will
simply call `array`. `Worklet2` is given `array` as a `FieldInOut` to
modify its values. Finally, `Worklet3` is given `array` as a `FieldIn`. It
is clear that for the computation to be correct, the three worklets _must_
execute in the correct order of `Worklet1`, `Worklet2`, and `Worklet3`.

The problem is that if `Worklet2` and `Worklet3` are both scheduled before
`Worklet1` finishes, the order they are executed could be arbitrary. Let us
say that `Worklet1` is invoked, and the invoke call returns before the
execution of `Worklet1` finishes.

The calling code immediately invokes `Worklet2`. Because `array` is already
locked by `Worklet1`, `Worklet2` does not execute right away. Instead, it
waits on a condition variable of `array` until it is free. But even though
the scheduling of `Worklet2` is blocked, the invoke returns because we are
scheduling asynchronously.

Likewise, the calling code then immediately calls invoke for `Worklet3`.
`Worklet3` similarly waits on the condition variable of `array` until it is
free.