Skip to content

Sst comm changes

Eisenhauer, Greg requested to merge github/fork/eisenhauer/SSTCommChanges into master

This PR consists of a set of changes that introduce a new "CPCommPattern" engine parameter. The prior peer-based communication pattern between reader and writer ranks will be the default for the moment and is requested with the value "peer". A new communication pattern, "min" in which essential control information (such as metadata, timestep release info, stream close notification, etc.) is exchanged directly only between writer rank 0 and reader rank 0. This information is then distributed as needed amongst the reader or writer ranks via specific MPI global operations. In the case of the reader, this requires the use of a global operation at blocking BeginStep which was not previously present. (Blocking global operations were already required if LatestAvailable or a timeout were specified.) For the writer, no additional global operations are introduced, however because event release only becomes known to non-zero ranks during the global operations that are part of EndStep(), events will generally be held longer than with peer communication. Additional complications occur with failure detection and handling. The main advantage of "min" communication mode is that it limits control plane connection requirements to a single connection held only by rank 0 (on each side). Note that DataPlanes, particularly the EVPath data plane have their own needs. Currently the RDMA data plane needs no connections, but the EVPath data plane's operation will cause connections to be made between a reader rank and any writer rank that has data that it requires. Connection needs are thus dependent upon actual data exchange patterns.

Merge request reports