The primary figure of merit for the ML4NSE (defined in detail in https://doi.org/10.1615/JMachLearnModelComput.2023048607) workflow benchmark is time-to-solution. Some useful secondary figures of merit will be:
- Exchange bandwidth (ingress) at the gateway node which multiplexes among input streams
- Scalability of input/output stream multiplexing – ideally, data rates at the gateway nodes on both sides of the application workflow will degrade linearly with the number of input/output streams.
An additional potential figure of merit that could demonstrate the robustness of the system would include any active guidance between the Compute and Services Clusters; the latency involved in control operations becomes crucial. Specifically, it's essential to assess the duration a compute job is held while disseminating new control information. Also, as additional input streams and output consumers are added, the effect on end-to-end time-to-solution could be affected.
