- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
The Intel MPI library 2019 uses PSM 2.x interface for Omni-Path fabric and PSM 1.x interface for QDR fabric. For very smaller message size, there is a switch-over in MPI implementation for Omni-Path network, but not for Infiniband interconnect network? Further, what are the default eager limits for intra-node and inter-node communication for Intel 2019? And specific control variables to tune these values? Thanks
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Starting IMPI 2019 we rely on OFI/libfabric infrastructure for low level transport: https://github.com/ofiwg/libfabric/
There are several knobs available for devices with verbs interface.
We use fi_rxm + fi_verbs combination for verbs RC path so you may check all available knobs on corresponding man pages:
- https://ofiwg.github.io/libfabric/v1.7.1/man/fi_rxm.7.html
- https://ofiwg.github.io/libfabric/v1.7.1/man/fi_verbs.7.html
The most relevant knobs to your question are:
FI_VERBS_INLINE_SIZE, FI_OFI_RXM_BUFFER_SIZE, FI_OFI_RXM_SAR_LIMIT, FI_VERBS_MR_CACHE_ENABLE
Would be great if you could clarify what kind of problem you would like to address. What kind of scenario you would like to tune?
BR,
Dmitry
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Dmitry,
Thanks for your quick reply. To be more precise, there are two questions.
1. Why there are two different default values of 16k and 256 Kb for messages transmitted via a rendezvous protocol (FI_OFI_RXM_BUFFER_SIZE, FI_OFI_RXM_SAR_LIMIT)?
2. For message size greater than 4096 byte, there seems an implementation difference only for Omni-Path networking system, and I'm unable to find this as a default value of any knob to know the real reason?
Regards,
![](/skins/images/D2683F18326913BBA0436CB7114DD569/responsive_peak/images/icon_anonymous_message.png)
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page