More options on substreams

Created by: jychoi-hpc

This is a feature request on aggregation (substreams).

I am wondering if we can add an option to set stride on aggregation (say, streamstride=Y).

Currently, if I run N processes and set X as substreams, rank 0 up to N/X-1 will be aggregated. I like to have an option to aggregate every Y-th processes (i.e., rank 0, Y-1, 2*Y-1, etc).

This will be helpful on Summit (and with SSD). Especially, XGC re-orders ranks and currently it is impossible to set to write XGC restart data by using one aggregator per node.

Any comment or suggestion will be appreciated.