topic significance/meaning of zero byte MPI messages (APS message profiling data) in Intel® MPI Library

significance/meaning of zero byte MPI messages (APS message profiling data)

psing51 — Wed, 15 Jan 2020 07:58:32 GMT

Hi,
I recently tried the APS tool to capture the message details (size and amount) for the WRF application on the intel 8280 (opa).
We launch 1 process per core, and here is the data -
nodes,Message_size(B),Volume(MB),Volume(%),Transfers,Time(sec),Time(%)
1,0,0,0,58099903,3988.14,97.05
2,0,0,0,219491539,7554.45,96.19
4,0,0,0,850730419,15073.44,96.02

It seems that significant amount of time is spent in the transfer of these 0 byte messages and with more number of nodes, the amount of messages increases. Could you please help me in understanding following-

Q: The significance of these 0 bytes messages and How are they related to MPI communication protocol?

I guess aps collects messages transferred between all processes (inter node + intra node), so
Q:Is there a way to check (from aps) that how much of these messages were transmitted to the network? (inter node messages - for 2 and 4 node runs)

Hi Puneet,

PrasanthD_intel — Thu, 16 Jan 2020 05:52:15 GMT

Hi Puneet,

Thanks for reaching out to us. We are working on your issue. we will get back to you soon.

-Prasanth

Thank you for the reply. The

psing51 — Thu, 16 Jan 2020 06:08:00 GMT

Thank you for the reply.
WRF version 3.9.1.1 and the dataset was conus 2.5km.

Messages to a target include

jimdempseyatthecove — Thu, 16 Jan 2020 13:45:31 GMT

Messages to a target include a Tag in addition to data (if any). Thus you can pass status (information) via Tag as opposed to in the data blob. If I were to guess, I suspect that this is the cause of 0-byte messages. A guessed-at example might be a SYNC or Watchdog tickle, but the user application can do this as well.

Jim Dempsey

Hi Punnet,

Klaus-Dieter_O_Intel — Mon, 04 May 2020 16:55:12 GMT

Hi Punnet,

Actually APS just shows correct message sizes for senders only. For receiving ranks APS currently shows 0 byte messages. Engineering is working on a fix.

Klaus-Dieter

Re:significance/meaning of zero byte MPI messages ...

GouthamK_Intel — Fri, 18 Jun 2021 10:45:54 GMT

Hi Puneet,

We have raised a feature request, and our engineering team is working on it. However, we cannot comment anything about the timeline when that feature gets available.

Could you please let us know if there is anything else that we can help you with?

If no, let us know if we can close this thread from our end.

Have a Good day.

Thanks & Regards

Goutham

Re:significance/meaning of zero byte MPI messages ...

GouthamK_Intel — Tue, 29 Jun 2021 05:58:51 GMT

Hi,

As we have not heard back from you, we are considering that we have answered all your queries. So we will no longer respond to this thread. If you require additional assistance from Intel, please start a new thread. Any further interaction in this thread will be considered community only

Have a Good day!

Thanks & Regards

Goutham