DataMan Bidirectional IP Address Assignment
Created by: stevenwalton
Currently DataMan only takes in the IP Address of the reader, or where the data is being sent to. It would be helpful for the other direction to be possible to. That readers could get the data by only knowing where the writer is.
Here's an example of where this would be useful:
I am trying to send data from one cluster to another cluster. A more straightforward way to do this would have a reader on loginNodeA receive the data from computeNodeA_{0,1,...,N}. The reader from loginNodeA would then send the data to loginNodeB and then loginNodeB would send the data to computeNodeB_{0,1,...,M}
In this case it would be extremely cumbersome to determine the M ip addresses needed. Instead DataMan should only need to know the address of what is sending the data, and the receivers can gather the data they need.
This also has a problem with the fact that the ports increase. Eg: If I run mpirun -n 5 someDataManWriter
and tell DataMan to run on port 200, then this will create 5 copies of someDataManWriter
who send on ports 20{0..4}. One for each instance of someDataManWriter
.