Running Griffin simulations with distributed mesh
@ahmedamin1 has requested that PyGriffin be configured to run with distributed mesh support. Currently, PyARC/PyGriffin relies on ReplicatedMesh to run its simulations, which basically copies the entire mesh onto each MPI process to execute. A more memory-efficient approach to run Griffin would be to use a distributed mesh. The easiest way to do this is to add the --distributed-mesh
flag to the Griffin execution line, and this basically copies the entire mesh onto each process and deletes the mesh regions that the process does not "own" after mesh partitioning. An alternative approach would be to use mesh splitting, which partitions the mesh beforehand and creates a separate "distributed" mesh file that each process will read during the MOOSE app execution. The benefit of doing this is that a single process does not have to read the entire mesh onto a single process. The command lines for doing this would be:
mpiexec -n $NPROC griffin-opt -i griffin_mesh.i griffin_options.i isoxml.i --split-mesh $NPROC --split-file split_mesh.cpr mpiexec -n $NPROC griffin-opt -i griffin_mesh.i griffin_options.i isoxml.i --use-split --split-file split_mesh.cpr
For more information about the difference between mesh splitting and calling distributed meshes directly, this link is a good reference: https://mooseframework.inl.gov/syntax/Mesh/
I think for now, if we support the --distributed-mesh option, that would be sufficient. If users ask for support with mesh splitting, we can add that down the line. To add distributed mesh support, we would basically need to update the PyARC schema, the PyGriffin execution command, and update PyGriffinConnect to tell PyGriffin to run Griffin with a distributed mesh.