Intel MPI 2021.7 on cluster

[Frank_R_1]

Dear support,

We have an Intel cluster with following installed os and drivers
CentOS Linux release 7.9.2009 (Core)
libibverbs-utils-22.4-6.el7_9.x86_64
libibverbs-22.4-6.el7_9.x86_64

Used library is Intel MPI 2021.7

Calling on two hosts the example ./IMB-MPI1 allreduce

./mpiexec.hydra -np 32 -hosts n17,n18 -genv UCX_TLS "ud_v,sm,self" -genv FI_PROVIDER=mlx -genv I_MPI_HYDRA_DEBUG 500 -genv I_MPI_FABRICS shm:ofi ./IMB-MPI1 allreduce

we get the output:
[1678203216.872833] [n18:604 :0] select.c:434 UCX ERROR no active messages transport to <no debug data>: posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, cma/memory - no am bcopy
[1678203216.872825] [n18:605 :0] select.c:434 UCX ERROR no active messages transport to <no debug data>: posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, cma/memory - no am bcopy
[1678203216.872829] [n18:610 :0] select.c:434 UCX ERROR no active messages transport to <no debug data>: posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, cma/memory - no am bcopy
[1678203216.872891] [n18:608 :0] select.c:434 UCX ERROR no active messages transport to <no debug data>: posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, cma/memory - no am bcopy
[1678203216.872870] [n18:609 :0] select.c:434 UCX ERROR no active messages transport to <no debug data>: posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, cma/memory - no am bcopy
Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: Other MPI error, error stack:
MPIR_Init_thread(176)............:
MPID_Init(1538)..................:
MPIDI_OFI_mpi_init_hook(1601)....:
insert_addr_table_roots_only(451): OFI get address vector map failed
[1678203216.872865] [n18:611 :0] select.c:434 UCX ERROR no active messages transport to <no debug data>: posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, cma/memory - no am bcopy
Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: Other MPI error, error stack:
truncated...

With -genv FI_PROVIDER=tcp it works.

Running ./mpiexec.hydra -np 32 -hosts n17,n18 -genv UCX_TLS "ud_v,sm,self" -genv FI_PROVIDER=verbs -genv I_MPI_HYDRA_DEBUG 500 -genv I_MPI_FABRICS shm:ofi ./IMB-MPI1 allreduce

we get the output:
Abort(2139535) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: Other MPI error, error stack:
MPIR_Init_thread(176)........:
MPID_Init(1538)..............:
MPIDI_OFI_mpi_init_hook(1511):
open_fabric(2565)............:
find_provider(2683)..........: OFI fi_getinfo() failed (ofi_init.c:2683:find_provider:No data available)
Abort(2139535) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: Other MPI error, error stack:
MPIR_Init_thread(176)........:
MPID_Init(1538)..............:
MPIDI_OFI_mpi_init_hook(1511):
open_fabric(2565)............:
find_provider(2683)..........: OFI fi_getinfo() failed (ofi_init.c:2683:find_provider:No data available)
Abort(2139535) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: Other MPI error, error stack:
MPIR_Init_thread(176)........:
MPID_Init(1538)..............:
MPIDI_OFI_mpi_init_hook(1511):
open_fabric(2565)............:
find_provider(2683)..........: OFI fi_getinfo() failed (ofi_init.c:2683:find_provider:No data available)
truncated...

Installed packets
elfutils-default-yama-scope-0.176-5.el7.noarch
pyparsing-1.5.6-9.el7.noarch
tuned-2.11.0-12.el7_9.noarch
fontpackages-filesystem-1.44-8.el7.noarch
abrt-addon-kerneloops-2.1.11-60.el7.centos.x86_64
xml-commons-resolver-1.2-15.el7.noarch
abrt-addon-pstoreoops-2.1.11-60.el7.centos.x86_64
ucx-knem-1.8.0-1.49224.x86_64
mesa-libGLU-9.0.0-4.el7.x86_64
cal10n-0.7.7-4.el7.noarch
librdmacm-22.4-6.el7_9.x86_64
libsepol-2.5-10.el7.x86_64
libarchive-3.1.2-14.el7_7.x86_64
kbd-misc-1.15.5-16.el7_9.noarch
adobe-mappings-cmap-20171205-3.el7.noarch
iowatcher-1.0-6.el7.x86_64
swig-2.0.10-5.el7.x86_64
libattr-2.4.46-13.el7.x86_64
libnfsidmap-0.25-19.el7.x86_64
perl-5.16.3-299.el7_9.x86_64
urw-base35-z003-fonts-20170801-10.el7.noarch
libffi-3.0.13-19.el7.x86_64
libproxy-0.4.11-11.el7.x86_64
graphviz-2.30.1-22.el7.x86_64
hardlink-1.0-19.el7.x86_64
cyrus-sasl-lib-2.1.26-24.el7_9.x86_64
dapl-2.1.5-3.el7.x86_64
lua-5.1.4-15.el7.x86_64
dracut-config-rescue-033-572.el7.x86_64
postfix-2.10.1-9.el7.x86_64
kpartx-0.4.9-136.el7_9.x86_64
libjpeg-turbo-1.2.90-8.el7.x86_64
libxcb-1.13-1.el7.x86_64
krb5-libs-1.15.1-55.el7_9.x86_64
groff-base-1.22.2-8.el7.x86_64
libXtst-1.2.3-1.el7.x86_64
python-rtslib-2.1.74-1.el7_9.noarch
grub2-tools-2.02-0.87.0.2.el7.centos.11.x86_64
slang-2.2.4-11.el7.x86_64
e2fsprogs-1.42.9-19.el7.x86_64
gpm-libs-1.20.7-6.el7.x86_64
kpatch-0.6.1-6.el7.noarch
kernel-tools-3.10.0-1160.81.1.el7.x86_64
perl-Pod-Perldoc-3.20-4.el7.noarch
libxslt-1.1.28-6.el7.x86_64
libsysfs-2.1.0-16.el7.x86_64
sysstat-10.1.5-20.el7_9.x86_64
perl-Storable-2.45-3.el7.x86_64
tomcat-servlet-3.0-api-7.0.76-16.el7_9.noarch
libutempter-1.1.6-4.el7.x86_64
bash-completion-2.1-8.el7.noarch
yumex-3.0.17-1.el7.noarch
perl-Carp-1.26-244.el7.noarch
wget-1.14-18.el7_6.1.x86_64
vim-enhanced-7.4.629-8.el7_9.x86_64
polkit-pkla-compat-0.1-4.el7.x86_64
rdate-1.4-25.el7.x86_64
libconfig-1.4.9-5.el7.x86_64
libmpc-1.0.1-3.el7.x86_64
policycoreutils-2.5-34.el7.x86_64
libglvnd-1.0.1-0.8.git5baa1e5.el7.x86_64
tcl-8.5.13-8.el7.x86_64
cyrus-sasl-plain-2.1.26-24.el7_9.x86_64
libxshmfence-1.2-1.el7.x86_64
perl-Pod-Escapes-1.04-299.el7_9.noarch
centos-indexhtml-7-9.el7.centos.noarch
boost-thread-1.53.0-28.el7.x86_64
iwl2000-firmware-18.168.6.1-80.el7_9.noarch
perl-IO-Compress-2.061-2.el7.noarch
iwl105-firmware-18.168.6.1-80.el7_9.noarch
ipset-libs-7.1-1.el7.x86_64
libglvnd-egl-1.0.1-0.8.git5baa1e5.el7.x86_64
iwl5150-firmware-8.24.2.2-80.el7_9.noarch
sed-4.2.2-7.el7.x86_64
libdb-utils-5.3.21-25.el7.x86_64
python-libs-2.7.5-92.el7_9.x86_64
libselinux-utils-2.5-15.el7.x86_64
ncurses-5.9-14.20130511.el7_4.x86_64
libfprint-0.8.2-1.el7.x86_64
at-spi2-core-2.28.0-1.el7.x86_64
python-six-1.9.0-2.el7.noarch
pam-1.1.8-23.el7.x86_64
glib-networking-2.56.1-1.el7.x86_64
gzip-1.5-11.el7_9.x86_64
selinux-policy-targeted-3.13.1-268.el7_9.2.noarch
samba-common-4.10.16-20.el7_9.noarch
libssh2-1.8.0-4.el7.x86_64
pygobject2-2.28.6-11.el7.x86_64
nss-softokn-freebl-3.79.0-4.el7_9.x86_64
dbus-libs-1.10.24-15.el7.x86_64
python-chardet-2.2.1-3.el7.noarch
pango-1.42.4-4.el7_7.x86_64
java-1.8.0-openjdk-headless-1.8.0.352.b08-2.el7_9.x86_64
stax2-api-3.1.1-10.el7.noarch
unit-api-1.0-3.el7.noarch
dejavu-fonts-common-2.33-6.el7.noarch
samba-client-4.10.16-20.el7_9.x86_64
NetworkManager-libnm-1.18.8-2.el7_9.x86_64
pyxattr-0.5.1-5.el7.x86_64
uom-lib-1.0.1-5.el7.noarch
kernel-devel-3.10.0-1160.76.1.el7.x86_64
libselinux-python-2.5-15.el7.x86_64
avalon-framework-4.3-10.el7.noarch
rdma-core-22.4-6.el7_9.x86_64
plymouth-core-libs-0.8.9-0.34.20140113.el7.centos.x86_64
fipscheck-1.4.1-6.el7.x86_64
jsr-311-1.1.1-6.el7.noarch
gd-2.0.35-27.el7_9.x86_64
bzip2-libs-1.0.6-13.el7.x86_64
expect-5.45-14.el7_1.x86_64
dracut-033-572.el7.x86_64
xmlrpc-c-client-1.32.5-1905.svn2451.el7.x86_64
gawk-4.0.2-4.el7_3.1.x86_64
urw-base35-nimbus-roman-fonts-20170801-10.el7.noarch
libmount-2.23.2-65.el7_9.1.x86_64
python-pycurl-7.19.0-19.el7.x86_64
libcap-ng-0.7.5-4.el7.x86_64
urw-base35-gothic-fonts-20170801-10.el7.noarch
abrt-cli-2.1.11-60.el7.centos.x86_64
rpm-libs-4.11.3-48.el7_9.x86_64
harfbuzz-1.7.5-2.el7.x86_64
libgcrypt-1.5.3-14.el7.x86_64
infiniband-diags-2.1.0-1.el7.x86_64
libstoragemgmt-1.8.1-2.el7_9.x86_64
usbutils-007-5.el7.x86_64
keyutils-libs-1.5.8-3.el7.x86_64
libibverbs-utils-22.4-6.el7_9.x86_64
NetworkManager-team-1.18.8-2.el7_9.x86_64
iscsi-initiator-utils-6.2.0.874-22.el7_9.x86_64
libndp-1.2-9.el7.x86_64
libreport-plugin-mailx-2.1.11-53.el7.centos.x86_64
kmod-libs-20-28.el7.x86_64
libXi-1.7.9-1.el7.x86_64
smartmontools-7.0-2.el7.x86_64
microcode_ctl-2.1-73.15.el7_9.x86_64
pyldb-1.5.4-2.el7.x86_64
libXxf86vm-1.1.4-1.el7.x86_64
ledmon-0.92-1.el7.x86_64
pixman-0.34.0-1.el7.x86_64
libsss_nss_idmap-1.16.5-10.el7_9.14.x86_64
mesa-libGL-18.3.4-12.el7_9.x86_64
vim-minimal-7.4.629-8.el7_9.x86_64
lcms2-2.6-3.el7.x86_64
perf-3.10.0-1160.81.1.el7.x86_64
libtiff-4.0.3-35.el7.x86_64
openssh-7.4p1-22.el7_9.x86_64
ustr-1.0.4-16.el7.x86_64
yum-langpacks-0.4.2-7.el7.noarch
perl-Time-HiRes-1.9725-3.el7.x86_64
pygtk2-2.24.0-9.el7.x86_64
libgomp-4.8.5-44.el7.x86_64
latencytop-tui-0.5-13.el7.x86_64
perl-Pod-Simple-3.28-4.el7.noarch
pcp-selinux-4.3.2-13.el7_9.x86_64
sg3_utils-1.37-19.el7.x86_64
libogg-1.3.0-7.el7.x86_64
time-1.7-45.el7.x86_64
attr-2.4.46-13.el7.x86_64
nfs-utils-1.3.0-0.68.el7.2.x86_64
bridge-utils-1.5-9.el7.x86_64
libref_array-0.1.5-32.el7.x86_64
dhclient-4.2.5-83.el7.centos.1.x86_64
libnl-1.1.4-3.el7.x86_64
libreport-plugin-mantisbt-2.1.11-53.el7.centos.x86_64
iw-4.3-2.el7.x86_64
binutils-2.27-44.base.el7_9.1.x86_64
python-pyudev-0.15-9.el7.noarch
gpg-pubkey-f4a80eb5-53a7ff4b
libtheora-1.1.1-8.el7.x86_64
iwl3945-firmware-15.32.2.9-80.el7_9.noarch
perl-DBD-SQLite-1.39-3.el7.x86_64
iwl5000-firmware-8.83.5.1_1-80.el7_9.noarch
libglvnd-glx-1.0.1-0.8.git5baa1e5.el7.x86_64
coreutils-8.22-24.el7_9.2.x86_64
jasper-libs-1.900.1-33.el7.x86_64
glibc-2.17-326.el7_9.x86_64
libdwarf-20130207-4.el7.x86_64
libreport-plugin-ureport-2.1.11-53.el7.centos.x86_64
dejavu-sans-fonts-2.33-6.el7.noarch
glusterfs-6.0-61.el7.x86_64
file-5.11-37.el7.x86_64
kmod-rshim-1.18-0.gb99e894.rhel7u9.x86_64
libtirpc-0.2.4-0.16.el7.x86_64
device-mapper-persistent-data-0.8.5-3.el7_9.2.x86_64
libicu-50.2-4.el7_7.x86_64
alsa-tools-firmware-1.1.0-1.el7.x86_64
gettext-libs-0.19.8.1-3.el7.x86_64
samba-client-libs-4.10.16-20.el7_9.x86_64
xdg-utils-1.1.0-0.17.20120809git.el7.noarch
nss-sysinit-3.79.0-4.el7_9.x86_64
libthai-0.1.14-9.el7.x86_64
python-kitchen-1.1.1-5.el7.noarch
kernel-3.10.0-1160.76.1.el7.x86_64
mft-4.15.1-9.x86_64
abrt-2.1.11-60.el7.centos.x86_64
python-javapackages-3.4.1-11.el7.noarch
langtable-0.0.31-4.el7.noarch
java-1.8.0-openjdk-1.8.0.352.b08-2.el7_9.x86_64
geronimo-jms-1.1.1-19.el7.noarch
pyliblzma-0.5.3-11.el7.x86_64
bind-utils-9.11.4-26.P2.el7_9.10.x86_64
ucx-devel-1.8.0-1.49224.x86_64
xerces-j2-2.11.0-17.el7_0.noarch
python-iniparse-0.4-9.el7.noarch
ncurses-base-5.9-14.20130511.el7_4.noarch
uom-systems-0.7-1.el7.noarch
python2-futures-3.1.1-5.el7.noarch
adwaita-cursor-theme-3.28.0-1.el7.noarch
apache-commons-logging-1.1.2-7.el7.noarch
python-slip-0.4.0-4.el7.noarch
expat-2.1.0-15.el7_9.x86_64
joda-time-2.2-3.tzdata2013c.el7.noarch
libibverbs-22.4-6.el7_9.x86_64
slf4j-1.7.4-4.el7_4.noarch
libXt-1.1.5-3.el7.x86_64
libselinux-2.5-15.el7.x86_64
quota-nls-4.01-19.el7.noarch
centos-logos-70.0.6-3.el7.centos.noarch
opa-basic-tools-10.9.0.0.204-1.el7.x86_64
mailx-12.5-19.el7.x86_64
ibutils-libs-1.5.7-14.el7.x86_64
libpaper-1.1.24-9.el7.x86_64
readline-6.2-11.el7.x86_64
libuser-0.60-9.el7.x86_64
libXxf86misc-1.0.3-7.1.el7.x86_64
libacl-2.2.51-15.el7.x86_64
perl-macros-5.16.3-299.el7_9.x86_64
GeoIP-1.5.0-14.el7.x86_64
urw-base35-c059-fonts-20170801-10.el7.noarch
json-c-0.11-4.el7_0.x86_64
libldb-1.5.4-2.el7.x86_64
fcoe-utils-1.0.32-2.el7_6.x86_64
python-urlgrabber-3.10-10.el7.noarch
urw-base35-nimbus-sans-fonts-20170801-10.el7.noarch
boost-date-time-1.53.0-28.el7.x86_64
urw-base35-fonts-20170801-10.el7.noarch
libtar-1.2.11-29.el7.x86_64
libpath_utils-0.2.1-32.el7.x86_64
ibutils-1.5.7-14.el7.x86_64
libgpg-error-1.12-3.el7.x86_64
gnupg2-2.0.22-5.el7_5.x86_64
glusterfs-rdma-6.0-61.el7.x86_64
jansson-2.10-1.el7.x86_64
libstoragemgmt-python-1.8.1-2.el7_9.noarch
authconfig-6.2.8-30.el7.x86_64
qperf-0.4.9-3.el7.x86_64
tcp_wrappers-libs-7.6-77.el7.x86_64
PyYAML-3.10-11.el7.x86_64
iwpmd-22.4-6.el7_9.x86_64
libidn-1.28-4.el7.x86_64
iscsi-initiator-utils-iscsiuio-6.2.0.874-22.el7_9.x86_64
psacct-6.6.1-13.el7.x86_64
device-mapper-multipath-0.4.9-136.el7_9.x86_64
p11-kit-0.23.5-3.el7.x86_64
biosdevname-0.7.3-2.el7.x86_64
libXext-1.3.3-3.el7.x86_64
libmnl-1.0.3-7.el7.x86_64
libXdamage-1.1.4-4.1.el7.x86_64
dhcp-common-4.2.5-83.el7.centos.1.x86_64
crda-3.18_2018.05.31-4.el7.x86_64
libXrandr-1.5.1-2.el7.x86_64
grub2-common-2.02-0.87.0.2.el7.centos.11.noarch
cronie-1.4.11-24.el7_9.x86_64
irqbalance-1.0.7-12.el7.x86_64
giflib-4.1.6-9.el7.x86_64
grub2-pc-modules-2.02-0.87.0.2.el7.centos.11.noarch
libverto-0.2.5-4.el7.x86_64
dmidecode-3.2-5.el7_9.1.x86_64
cryptsetup-2.0.3-6.el7.x86_64
lsscsi-0.27-6.el7.x86_64
libsss_idmap-1.16.5-10.el7_9.14.x86_64
perl-HTTP-Tiny-0.033-3.el7.noarch
oprofile-0.9.9-27.el7.x86_64
bpftool-3.10.0-1160.81.1.el7.x86_64
perl-Encode-2.51-7.el7.x86_64
device-mapper-libs-1.02.170-6.el7_9.5.x86_64
man-db-2.6.3-11.el7.x86_64
libpipeline-1.2.3-3.el7.x86_64
kernel-devel-3.10.0-1160.81.1.el7.x86_64
libsemanage-2.5-14.el7.x86_64
tzdata-java-2022g-1.el7.noarch
perl-Exporter-5.68-3.el7.noarch
pdsh-rcmd-ssh-2.34-5.el7.x86_64
libdaemon-0.14-7.el7.x86_64
python2-pyxdg-0.25-8.el7.noarch
perl-Scalar-List-Utils-1.27-248.el7.x86_64
lvm2-libs-2.02.187-6.el7_9.5.x86_64
mtr-0.85-7.el7.x86_64
perl-File-Path-2.09-2.el7.noarch
libX11-1.6.7-4.el7_9.x86_64
dstat-0.7.2-12.el7.noarch
perl-Getopt-Long-2.40-3.el7.noarch
linux-firmware-20200421-80.git78c0348.el7_9.noarch
hunspell-en-0.20121024-6.el7.noarch
firewalld-filesystem-0.6.3-13.el7_9.noarch
satyr-0.13-15.el7.x86_64
kernel-3.10.0-1160.59.1.el7.x86_64
nano-2.3.1-10.el7.x86_64
tar-1.26-35.el7.x86_64
traceroute-2.0.22-2.el7.x86_64
libunistring-0.9.3-9.el7.x86_64
openssh-server-7.4p1-22.el7_9.x86_64
dosfstools-3.0.20-10.el7.x86_64
avahi-libs-0.6.31-20.el7.x86_64
libbasicobjects-0.1.1-32.el7.x86_64
libgfortran-4.8.5-44.el7.x86_64
libnfnetlink-1.0.1-4.el7.x86_64
gcc-4.8.5-44.el7.x86_64
dbus-glib-0.100-7.el7.x86_64
numactl-libs-2.0.12-5.el7.x86_64
gcc-gfortran-4.8.5-44.el7.x86_64
ivtv-firmware-20080701-26.el7.noarch
p11-kit-trust-0.23.5-3.el7.x86_64
wpa_supplicant-2.6-12.el7_9.2.x86_64
libpciaccess-0.14-1.el7.x86_64
sg3_utils-libs-1.37-19.el7.x86_64
os-prober-1.58-9.el7.x86_64
kernel-devel-3.10.0-1160.59.1.el7.x86_64
words-3.0-22.el7.noarch
libvorbis-1.3.3-8.el7.1.x86_64
centos-release-7-9.2009.1.el7.centos.x86_64
perl-Net-Daemon-0.48-5.el7.noarch
libcgroup-tools-0.41-21.el7.x86_64
perl-DBI-1.627-4.el7.x86_64
dmraid-1.0.0.rc16-28.el7.x86_64
less-458-9.el7.x86_64
chkconfig-1.7.6-1.el7.x86_64
cairo-1.15.12-4.el7.x86_64
glibc-common-2.17-326.el7_9.x86_64
libstdc++-4.8.5-44.el7.x86_64
xz-libs-5.2.2-2.el7_9.x86_64
mozjs17-17.0.0-20.el7.x86_64
abrt-libs-2.1.11-60.el7.centos.x86_64
adwaita-icon-theme-3.28.0-1.el7.noarch
glusterfs-client-xlators-6.0-61.el7.x86_64
file-libs-5.11-37.el7.x86_64
usb_modeswitch-data-20170806-1.el7.noarch
libgusb-0.2.9-1.el7.x86_64
libreport-cli-2.1.11-53.el7.centos.x86_64
shared-mime-info-1.8-5.el7.x86_64
glibc-headers-2.17-326.el7_9.x86_64
libpng-1.5.13-8.el7.x86_64
dconf-0.28.0-4.el7.x86_64
gtk3-3.22.30-8.el7_9.x86_64
mpi-selector-1.0.3-1.49224.x86_64
python-decorator-3.4.0-3.el7.noarch
gnutls-3.3.29-9.el7_6.x86_64
cracklib-dicts-2.9.0-11.el7.x86_64
gdb-7.6.1-120.el7.x86_64
at-3.1.13-25.el7_9.x86_64
e2fsprogs-libs-1.42.9-19.el7.x86_64
rest-0.8.1-2.el7.x86_64
hicolor-icon-theme-0.12-7.el7.noarch
kernel-devel-3.10.0-1160.71.1.el7.x86_64
xorg-x11-fonts-Type1-7.5-9.el7.noarch
nspr-4.34.0-3.1.el7_9.x86_64
gobject-introspection-1.56.1-1.el7.x86_64
libwbclient-4.10.16-20.el7_9.x86_64
curl-7.29.0-59.el7_9.1.x86_64
desktop-file-utils-0.23-2.el7.x86_64
bind-libs-9.11.4-26.P2.el7_9.10.x86_64
elfutils-libs-0.176-5.el7.x86_64
lm_sensors-libs-3.4.0-8.20160601gitf9185e5.el7.x86_64
nss-3.79.0-4.el7_9.x86_64
hwdata-0.252-9.7.el7.x86_64
NetworkManager-1.18.8-2.el7_9.x86_64
abrt-tui-2.1.11-60.el7.centos.x86_64
abrt-addon-python-2.1.11-60.el7.centos.x86_64
elfutils-0.176-5.el7.x86_64
yum-plugin-fastestmirror-1.1.31-54.el7_8.noarch
kmod-kvdo-6.1.3.23-5.el7.x86_64
dracut-network-033-572.el7.x86_64
vdo-6.1.3.23-5.el7.x86_64
mesa-libEGL-18.3.4-12.el7_9.x86_64
libcroco-0.6.12-6.el7_9.x86_64
xfsprogs-4.5.0-22.el7.x86_64
tcsh-6.18.01-17.el7_9.1.x86_64
nfs4-acl-tools-0.3.3-21.el7.x86_64
man-pages-overrides-7.9.0-1.el7.x86_64
nettle-2.7.1-9.el7_9.x86_64
iwl135-firmware-18.168.6.1-80.el7_9.noarch
iwl1000-firmware-39.31.5.1-80.el7_9.noarch
iwl4965-firmware-228.61.2.24-80.el7_9.noarch
iwl3160-firmware-25.30.13.0-80.el7_9.noarch
dbus-1.10.24-15.el7.x86_64
mstflint-4.14.0-3.49224.x86_64
python-kmod-0.9-4.el7.x86_64
jbigkit-libs-2.0-11.el7.x86_64
filesystem-3.2-25.el7.x86_64
javapackages-tools-3.4.1-11.el7.noarch
gtk2-2.24.31-1.el7.x86_64
libreport-filesystem-2.1.11-53.el7.centos.x86_64
python-ethtool-0.8-8.el7.x86_64
lldpad-1.0.1-7.git036e314.el7_9.x86_64
apache-commons-lang-2.6-15.el7.noarch
basesystem-10.0-7.el7.centos.noarch
ucx-1.8.0-1.49224.x86_64
python-linux-procfs-0.4.11-4.el7.noarch
samba-libs-4.10.16-20.el7_9.x86_64
xalan-j2-2.7.1-23.el7.noarch
python-tdb-1.3.18-1.el7.x86_64
systemd-sysv-219-78.el7_9.7.x86_64
si-units-0.6.5-1.el7.noarch
emacs-filesystem-24.3-23.el7.noarch
xfsdump-3.1.7-2.el7_9.x86_64
avalon-logkit-2.1-14.el7.noarch
python-urwid-1.1.1-3.el7.x86_64
bind-export-libs-9.11.4-26.P2.el7_9.10.x86_64
joda-convert-1.3-5.el7.noarch
libibumad-22.4-6.el7_9.x86_64
javassist-3.16.1-10.el7.noarch
pcre-8.32-17.el7.x86_64
python-firewall-0.6.3-13.el7_9.noarch
libXmu-1.1.2-2.el7.x86_64
librsvg2-2.40.20-1.el7.x86_64
info-5.1-5.el7.x86_64
pcp-conf-4.3.2-13.el7_9.x86_64
libtool-ltdl-2.4.2-22.el7_3.x86_64
popt-1.13-16.el7.x86_64
libXaw-1.0.13-4.el7.x86_64
libxkbcommon-0.7.1-3.el7.x86_64
grep-2.20-3.el7.x86_64
openjpeg2-2.3.1-3.el7_7.x86_64
pm-utils-1.4.1-27.el7.x86_64
libcap-2.22-11.el7.x86_64
libxml2-2.9.1-6.el7_9.6.x86_64
passwd-0.79-6.el7.x86_64
xorg-x11-server-utils-7.7-20.el7.x86_64
latencytop-0.5-13.el7.x86_64
libtalloc-2.1.16-1.el7.x86_64
libblkid-2.23.2-65.el7_9.1.x86_64
urw-base35-d050000l-fonts-20170801-10.el7.noarch
sqlite-3.7.17-8.el7_7.1.x86_64
libmodman-2.0.1-8.el7.x86_64
urw-base35-standard-symbols-ps-fonts-20170801-10.el7.noarch
cifs-utils-6.2-10.el7.x86_64
which-2.20-7.el7.x86_64
dyninst-9.3.1-3.el7.x86_64
libgs-9.25-5.el7.x86_64
fprintd-pam-0.8.1-2.el7.x86_64
libtdb-1.3.18-1.el7.x86_64
libini_config-1.3.1-32.el7.x86_64
opa-fastfabric-10.9.0.0.204-1.el7.x86_64
cgdcbxd-1.0.2-7.el7.x86_64
libnl3-3.2.28-4.el7.x86_64
gpgme-1.3.2-5.el7.x86_64
librdmacm-utils-22.4-6.el7_9.x86_64
mlocate-0.26-8.el7.x86_64
opa-libopamgt-10.9.0.0.204-1.el7.x86_64
pciutils-3.5.1-3.el7.x86_64
pcp-libs-4.3.2-13.el7_9.x86_64
libseccomp-2.3.1-4.el7.x86_64
audit-2.8.5-4.el7.x86_64
libaio-0.3.109-13.el7.x86_64
gssproxy-0.7.0-30.el7_9.x86_64
libXau-1.0.8-2.1.el7.x86_64
mcelog-144-10.94d853b2ea81.el7.x86_64
pciutils-libs-3.5.1-3.el7.x86_64
openldap-2.4.44-25.el7_9.x86_64
libXrender-0.9.10-1.el7.x86_64
kernel-devel-3.10.0-1160.80.1.el7.x86_64
rng-tools-6.3.1-5.el7.x86_64
dhcp-libs-4.2.5-83.el7.centos.1.x86_64
libXcomposite-0.4.4-4.1.el7.x86_64
kernel-3.10.0-1160.80.1.el7.x86_64
net-tools-2.0-0.25.20131004git.el7.x86_64
hunspell-en-US-0.20121024-6.el7.noarch
cronie-anacron-1.4.11-24.el7_9.x86_64
libXinerama-1.1.3-2.1.el7.x86_64
grub2-tools-minimal-2.02-0.87.0.2.el7.centos.11.x86_64
powertop-2.9-1.el7.x86_64
newt-0.52.15-4.el7.x86_64
grub2-pc-2.02-0.87.0.2.el7.centos.11.x86_64
perl-parent-0.225-244.el7.noarch
polkit-0.112-26.el7_9.1.x86_64
sssd-client-1.16.5-10.el7_9.14.x86_64
perl-Text-ParseWords-3.29-4.el7.noarch
util-linux-2.23.2-65.el7_9.1.x86_64
python-perf-3.10.0-1160.81.1.el7.x86_64
device-mapper-event-libs-1.02.170-6.el7_9.5.x86_64
kernel-headers-3.10.0-1160.81.1.el7.x86_64
perl-Filter-1.49-3.el7.x86_64
pdsh-2.34-5.el7.x86_64
shadow-utils-4.6-5.el7.x86_64
rsync-3.1.2-12.el7_9.x86_64
perl-threads-shared-1.43-6.el7.x86_64
device-mapper-event-1.02.170-6.el7_9.5.x86_64
qrencode-libs-3.4.1-3.el7.x86_64
pexpect-2.3-11.el7.noarch
perl-File-Temp-0.23.01-3.el7.noarch
virt-what-1.18-4.el7_9.1.x86_64
btrfs-progs-4.9.1-1.el7.x86_64
perl-Socket-2.010-5.el7.x86_64
cryptsetup-libs-2.0.3-6.el7.x86_64
lzo-2.06-8.el7.x86_64
vim-common-7.4.629-8.el7_9.x86_64
libnl3-cli-3.2.28-4.el7.x86_64
pcp-4.3.2-13.el7_9.x86_64
libreport-python-2.1.11-53.el7.centos.x86_64
zip-3.0-11.el7.x86_64
libwayland-server-1.15.0-1.el7.x86_64
lsof-4.87-6.el7.x86_64
pdsh-mod-dshgroup-2.34-5.el7.x86_64
libusbx-1.0.21-1.el7.x86_64
rfkill-0.4-10.el7.x86_64
mpfr-3.1.1-4.el7.x86_64
boost-system-1.53.0-28.el7.x86_64
cpp-4.8.5-44.el7.x86_64
sysvinit-tools-2.88-14.dsf.el7.x86_64
targetcli-2.1.53-1.el7_9.noarch
crontabs-1.11-6.20121102git.el7.noarch
rootfiles-8.1-11.el7.noarch
libquadmath-devel-4.8.5-44.el7.x86_64
libcollection-0.7.0-32.el7.x86_64
rpm-python-4.11.3-48.el7_9.x86_64
dbus-python-1.1.1-9.el7.x86_64
fuse-libs-2.9.2-11.el7.x86_64
libtasn1-4.10-1.el7.x86_64
openssl-1.0.2k-25.el7_9.x86_64
ethtool-4.8-10.el7.x86_64
kexec-tools-2.0.15-51.el7_9.3.x86_64
libdrm-2.4.97-2.el7.x86_64
libnetfilter_conntrack-1.0.6-1.el7_3.x86_64
kbd-1.15.5-16.el7_9.x86_64
perl-Data-Dumper-2.145-3.el7.x86_64
perl-PlRPC-0.2020-14.el7.noarch
hunspell-en-GB-0.20121024-6.el7.noarch
ipset-7.1-1.el7.x86_64
tcp_wrappers-7.6-77.el7.x86_64
libpng12-1.2.50-10.el7.x86_64
cairo-gobject-1.15.12-4.el7.x86_64
acl-2.2.51-15.el7.x86_64
python-slip-dbus-0.4.0-4.el7.noarch
cups-libs-1.6.3-51.el7.x86_64
make-3.82-24.el7.x86_64
python-2.7.5-92.el7_9.x86_64
usb_modeswitch-2.5.1-1.el7.x86_64
mesa-libglapi-18.3.4-12.el7_9.x86_64
gtk-update-icon-cache-3.22.30-8.el7_9.x86_64
colord-libs-1.3.4-2.el7.x86_64
iproute-4.11.0-30.el7.x86_64
atk-2.28.1-2.el7.x86_64
ebtables-2.0.10-16.el7.x86_64
cracklib-2.9.0-11.el7.x86_64
python-augeas-0.5.0-2.el7.noarch
libss-1.42.9-19.el7.x86_64
alsa-lib-1.1.8-1.el7.x86_64
kernel-3.10.0-1160.71.1.el7.x86_64
fxload-2002_04_11-16.el7.x86_64
xorg-x11-font-utils-7.5-21.el7.x86_64
systemd-219-78.el7_9.7.x86_64
libteam-1.29-3.el7.x86_64
gsettings-desktop-schemas-3.28.0-3.el7.x86_64
samba-common-libs-4.10.16-20.el7_9.x86_64
libcurl-7.29.0-59.el7_9.1.x86_64
yum-metadata-parser-1.1.4-10.el7.x86_64
bind-libs-lite-9.11.4-26.P2.el7_9.10.x86_64
grubby-8.28-26.el7.x86_64
nss-softokn-3.79.0-4.el7_9.x86_64
abrt-dbus-2.1.11-60.el7.centos.x86_64
abrt-addon-xorg-2.1.11-60.el7.centos.x86_64
abrt-addon-ccpp-2.1.11-60.el7.centos.x86_64
yum-3.4.3-168.el7.centos.noarch
initscripts-9.49.53-1.el7_9.1.x86_64
plymouth-scripts-0.8.9-0.34.20140113.el7.centos.x86_64
rasdaemon-0.4.1-37.el7.x86_64
ntpdate-4.2.6p5-29.el7.centos.2.x86_64
strace-4.24-6.el7.x86_64
hunspell-1.3.2-16.el7.x86_64
ntsysv-1.7.6-1.el7.x86_64
SDL-1.2.15-17.el7.x86_64
iwl2030-firmware-18.168.6.1-80.el7_9.noarch
iwl6000g2a-firmware-18.168.6.1-80.el7_9.noarch
epel-release-7-14.noarch
iwl100-firmware-39.31.5.1-80.el7_9.noarch
libepoxy-1.5.2-1.el7.x86_64
ca-certificates-2022.2.54-74.el7_9.noarch
setup-2.8.71-11.el7.noarch
bea-stax-api-1.2.0-9.el7.noarch
xml-commons-apis-1.4.01-16.el7.noarch
nss-tools-3.79.0-4.el7_9.x86_64
langtable-data-0.0.31-4.el7.noarch
python-schedutils-0.4-6.el7.x86_64
ucx-cma-1.8.0-1.49224.x86_64
uom-se-1.0.4-3.el7.noarch
abrt-retrace-client-2.1.11-60.el7.centos.x86_64
langtable-python-0.0.31-4.el7.noarch
guava-13.0-6.el7.noarch
urw-base35-fonts-common-20170801-10.el7.noarch
ncurses-libs-5.9-14.20130511.el7_4.x86_64
teamd-1.29-3.el7.x86_64
fipscheck-lib-1.4.1-6.el7.x86_64
libX11-common-1.6.7-4.el7_9.noarch
jackson-1.9.4-7.el7.noarch
opensm-libs-3.3.21-4.el7_9.x86_64
nss-pem-1.0.3-7.el7.x86_64
bash-4.2.46-35.el7_9.x86_64
xkeyboard-config-2.24-1.el7.noarch
ibacm-22.4-6.el7_9.x86_64
libuuid-2.23.2-65.el7_9.1.x86_64
parfait-0.5.4-2.el7.noarch
urw-base35-p052-fonts-20170801-10.el7.noarch
libdb-5.3.21-25.el7.x86_64
plymouth-0.8.9-0.34.20140113.el7.centos.x86_64
urw-base35-bookman-fonts-20170801-10.el7.noarch
audit-libs-2.8.5-4.el7.x86_64
abrt-addon-vmcore-2.1.11-60.el7.centos.x86_64
graphite2-1.3.10-1.el7_3.x86_64
rpm-4.11.3-48.el7_9.x86_64
opa-address-resolution-10.9.0.0.204-1.el7.x86_64
libtevent-0.9.39-1.el7.x86_64
abrt-console-notification-2.1.11-60.el7.centos.x86_64
pygpgme-0.3-9.el7.x86_64
libstoragemgmt-python-clibs-1.8.1-2.el7_9.x86_64
srp_daemon-22.4-6.el7_9.x86_64
xmlrpc-c-1.32.5-1905.svn2451.el7.x86_64
yum-utils-1.1.31-54.el7_8.noarch
libwayland-egl-1.15.0-1.el7.x86_64
diffutils-3.3-5.el7.x86_64
lshw-B.02.18-17.el7.x86_64
libXfixes-5.0.3-1.el7.x86_64
aic94xx-firmware-30-6.el7.noarch
libXft-2.3.2-2.el7.x86_64
libxml2-python-2.9.1-6.el7_9.6.x86_64
kernel-tools-libs-3.10.0-1160.81.1.el7.x86_64
gdbm-1.10-8.el7.x86_64
systemtap-runtime-4.0-13.el7.x86_64
theora-tools-1.1.1-8.el7.x86_64
libsmartcols-2.23.2-65.el7_9.1.x86_64
tcpdump-4.9.2-4.el7_7.1.x86_64
tzdata-2022g-1.el7.noarch
perl-Pod-Usage-1.63-3.el7.noarch
mariadb-libs-5.5.68-1.el7.x86_64
pcsc-lite-libs-1.8.8-8.el7.x86_64
openssh-clients-7.4p1-22.el7_9.x86_64
pycairo-1.8.10-8.el7.x86_64
perl-constant-1.27-2.el7.noarch
iprutils-2.4.17.1-3.el7_7.x86_64
perl-PathTools-3.40-5.el7.x86_64
vim-filesystem-7.4.629-8.el7_9.x86_64
keyutils-1.5.8-3.el7.x86_64
libreport-2.1.11-53.el7.centos.x86_64
log4j-1.2.17-18.el7_4.noarch
ed-1.9-4.el7.x86_64
bzip2-1.0.6-13.el7.x86_64
libreport-web-2.1.11-53.el7.centos.x86_64
sudo-1.8.23-10.el7_9.2.x86_64
setserial-2.17-33.el7.x86_64
hostname-3.13-3.el7_7.1.x86_64
iputils-20160308-10.el7.x86_64
autofs-5.0.7-116.el7_9.x86_64
yajl-2.0.4-4.el7.x86_64
mdadm-4.1-9.el7_9.x86_64
man-pages-3.53-5.el7.noarch
libSM-1.2.2-2.el7.x86_64
libcgroup-0.41-21.el7.x86_64
libgcc-4.8.5-44.el7.x86_64
perl-Compress-Raw-Bzip2-2.061-3.el7.x86_64
iwl7260-firmware-25.30.13.0-80.el7_9.noarch
iwl6050-firmware-41.28.5.1-80.el7_9.noarch
libcom_err-1.42.9-19.el7.x86_64
hesiod-3.2.1-3.el7.x86_64
libreport-centos-2.1.11-53.el7.centos.x86_64
zlib-1.2.7-20.el7_9.x86_64
pinentry-0.8.1-17.el7.x86_64
libreport-plugin-rhtsupport-2.1.11-53.el7.centos.x86_64
lz4-1.8.3-1.el7.x86_64
iptables-1.4.21-35.el7.x86_64
sos-3.9-5.el7.centos.11.noarch
mesa-libgbm-18.3.4-12.el7_9.x86_64
rsyslog-8.24.0-57.el7_9.3.x86_64
libevent-2.0.21-4.el7.x86_64
alsa-firmware-1.0.28-2.el7.noarch
systemd-libs-219-78.el7_9.7.x86_64
gettext-0.19.8.1-3.el7.x86_64
bind-license-9.11.4-26.P2.el7_9.10.noarch
json-glib-1.4.2-2.el7.x86_64
libestr-0.1.9-2.el7.x86_64
copy-jdk-configs-3.3-11.el7_9.noarch
abrt-python-2.1.11-60.el7.centos.x86_64
python-lxml-3.2.1-4.el7.x86_64
gdk-pixbuf2-2.36.12-3.el7.x86_64
latencytop-common-0.5-13.el7.x86_64
procps-ng-3.3.10-28.el7.x86_64
python-configobj-4.7.2-7.el7.noarch
objectweb-asm-3.3.1-9.el7.noarch
systemd-python-219-78.el7_9.7.x86_64
newt-python-0.52.15-4.el7.x86_64
javamail-1.4.6-8.el7.noarch
blktrace-1.0.5-9.el7.x86_64
apache-commons-io-2.4-12.el7.noarch
libXpm-3.5.12-1.el7.x86_64
kbd-legacy-1.15.5-16.el7_9.noarch
logrotate-3.8.6-19.el7.x86_64
quota-4.01-19.el7.x86_64
adobe-mappings-cmap-deprecated-20171205-3.el7.noarch
librsvg2-tools-2.40.20-1.el7.x86_64
adobe-mappings-pdf-20180407-1.el7.noarch
perl-libs-5.16.3-299.el7_9.x86_64
geoipupdate-2.5.0-1.el7.x86_64
setuptool-1.19.11-8.el7.x86_64
urw-base35-nimbus-mono-ps-fonts-20170801-10.el7.noarch
glib2-2.56.1-9.el7_9.x86_64
snappy-1.1.0-3.el7.x86_64
graphviz-tcl-2.30.1-22.el7.x86_64
openssl-libs-1.0.2k-25.el7_9.x86_64
pth-2.0.7-23.el7.x86_64
perftest-4.2-2.el7.x86_64
libyaml-0.1.4-11.el7_0.x86_64
findutils-4.5.11-6.el7.x86_64
device-mapper-multipath-libs-0.4.9-136.el7_9.x86_64
NetworkManager-tui-1.18.8-2.el7_9.x86_64
rpm-build-libs-4.11.3-48.el7_9.x86_64
chrony-3.4-1.el7.x86_64
libwayland-client-1.15.0-1.el7.x86_64
libXcursor-1.1.15-1.el7.x86_64
gmp-6.0.0-15.el7.x86_64
grub2-tools-extra-2.02-0.87.0.2.el7.centos.11.x86_64
libfastjson-0.99.4-3.el7.x86_64
parted-3.1-32.el7.x86_64
perl-podlators-2.5.1-3.el7.noarch
grub2-2.02-0.87.0.2.el7.centos.11.x86_64
augeas-libs-1.4.0-10.el7.x86_64
device-mapper-1.02.170-6.el7_9.5.x86_64
fribidi-1.0.2-1.el7_7.1.x86_64
perl-threads-1.87-4.el7.x86_64
kernel-3.10.0-1160.81.1.el7.x86_64
sgpio-1.2.0.10-13.el7.x86_64
pinfo-0.6.10-9.el7.x86_64
perl-Time-Local-1.2300-2.el7.noarch
kmod-20-28.el7.x86_64
iotop-0.6-4.el7.noarch
firewalld-0.6.3-13.el7_9.noarch
bc-1.06.95-13.el7.x86_64
libassuan-2.1.0-3.el7.x86_64
gpg-pubkey-352c64e5-52ae6884
lvm2-2.02.187-6.el7_9.5.x86_64
scl-utils-20130529-19.el7.x86_64
libedit-3.0-12.20121213cvs.el7.x86_64
libquadmath-4.8.5-44.el7.x86_64
libICE-1.0.9-9.el7.x86_64
tk-8.5.13-6.el7.x86_64
rpcbind-0.2.0-49.el7.x86_64
unzip-6.0-24.el7_9.x86_64
perl-Compress-Raw-Zlib-2.061-4.el7.x86_64
iwl6000g2b-firmware-18.168.6.1-80.el7_9.noarch
dmraid-events-1.0.0.rc16-28.el7.x86_64
libwayland-cursor-1.15.0-1.el7.x86_64
iwl6000-firmware-9.221.4.1-80.el7_9.noarch
elfutils-libelf-0.176-5.el7.x86_64
ttmkfdir-3.0.9-42.el7.x86_64
libfontenc-1.1.3-3.el7.x86_64
glusterfs-libs-6.0-61.el7.x86_64
cpio-2.11-28.el7.x86_64
lksctp-tools-1.0.17-2.el7.x86_64
xz-5.2.2-2.el7_9.x86_64
fprintd-0.8.1-2.el7.x86_64
fontconfig-2.13.0-4.3.el7.x86_64
glibc-devel-2.17-326.el7_9.x86_64
at-spi2-atk-2.26.2-1.el7.x86_64
freetype-2.8-14.el7_9.1.x86_64
pkgconfig-0.27.1-4.el7.x86_64
glusterfs-fuse-6.0-61.el7.x86_64
trousers-0.3.14-2.el7.x86_64
python-configshell-1.1.26-1.el7.noarch
libpwquality-1.2.3-5.el7.x86_64
libpcap-1.5.3-13.el7_9.x86_64
libsoup-2.62.2-2.el7.x86_64
selinux-policy-3.13.1-268.el7_9.2.noarch
libverto-libevent-0.2.5-4.el7.x86_64
nss-util-3.79.0-1.el7_9.x86_64
psmisc-22.20-17.el7.x86_64
python-gobject-base-3.22.0-1.el7_4.1.x86_64
libsmbclient-4.10.16-20.el7_9.x86_64
usermode-1.111-6.el7.x86_64
pytalloc-2.1.16-1.el7.x86_64

Installed modules:
Module Size Used by
rpcsec_gss_krb5 35549 0
auth_rpcgss 59415 1 rpcsec_gss_krb5
nfsv4 592234 1
dns_resolver 13140 1 nfsv4
nfs 270197 4 nfsv4
lockd 98048 1 nfs
grace 13515 1 lockd
fscache 64980 2 nfs,nfsv4
ib_isert 50874 0
iscsi_target_mod 295703 1 ib_isert
ib_srpt 52867 0
target_core_mod 347008 3 iscsi_target_mod,ib_srpt,ib_isert
ib_srp 56980 0
scsi_transport_srp 21112 1 ib_srp
scsi_tgt 20027 1 scsi_transport_srp
rpcrdma 221211 0
rdma_ucm 26958 0
ib_iser 47950 0
ib_umad 22093 0
rdma_cm 64491 6 rpcrdma,ib_srp,ib_iser,ib_srpt,rdma_ucm,ib_isert
ib_ipoib 115195 0
iw_cm 43514 1 rdma_cm
libiscsi 57233 1 ib_iser
scsi_transport_iscsi 108101 2 ib_iser,libiscsi
ib_cm 53122 4 rdma_cm,ib_srp,ib_srpt,ib_ipoib
sunrpc 366617 12 nfs,rpcsec_gss_krb5,auth_rpcgss,lockd,nfsv4,rpcrdma
mlx4_ib 183097 0
ib_uverbs 106304 2 mlx4_ib,rdma_ucm
ib_core 263545 13 rdma_cm,ib_cm,iw_cm,rpcrdma,mlx4_ib,ib_srp,ib_iser,ib_srpt,ib_umad,ib_uverbs,rdma_ucm,ib_ipoib,ib_isert
dm_mirror 22326 0
dm_region_hash 20813 1 dm_mirror
dm_log 18411 2 dm_region_hash,dm_mirror
dm_mod 128595 2 dm_log,dm_mirror
iTCO_wdt 13572 0
iTCO_vendor_support 13242 1 iTCO_wdt
sb_edac 32114 0
intel_powerclamp 14451 0
coretemp 13444 0
intel_rapl 19451 0
iosf_mbi 15582 1 intel_rapl
kvm 653928 0
ipmi_ssif 29595 0
irqbypass 13503 1 kvm
crc32_pclmul 13133 0
ghash_clmulni_intel 13273 0
cryptd 21190 1 ghash_clmulni_intel
sg 40719 0
pcspkr 12718 0
i2c_i801 22736 0
joydev 17389 0
lpc_ich 21086 0
mei_me 32848 0
mei 95246 1 mei_me
ioatdma 71905 0
ipmi_si 59965 0
ipmi_devintf 17459 0
ipmi_msghandler 56728 3 ipmi_ssif,ipmi_devintf,ipmi_si
ip_tables 27126 0
ext4 592325 2
mbcache 14958 1 ext4
jbd2 107486 1 ext4
mlx4_en 121065 0
sd_mod 46281 4
crc_t10dif 12912 2 target_core_mod,sd_mod
crct10dif_generic 12647 0
mgag200 40946 1
drm_kms_helper 186531 1 mgag200
syscopyarea 12529 1 drm_kms_helper
sysfillrect 12701 1 drm_kms_helper
sysimgblt 12640 1 drm_kms_helper
fb_sys_fops 12703 1 drm_kms_helper
ttm 100769 1 mgag200
isci 141644 0
ahci 34056 0
drm 468454 4 ttm,drm_kms_helper,mgag200
mlx4_core 324088 2 mlx4_en,mlx4_ib
libsas 79001 1 isci
libahci 31992 1 ahci
scsi_transport_sas 41224 2 isci,libsas
igb 223919 0
megaraid_sas 163773 3
libata 247190 3 ahci,libahci,libsas
crct10dif_pclmul 14307 1
crct10dif_common 12595 3 crct10dif_pclmul,crct10dif_generic,crc_t10dif
crc32c_intel 22094 0
ptp 23551 2 igb,mlx4_en
devlink 60067 3 mlx4_en,mlx4_ib,mlx4_core
pps_core 19057 1 ptp
drm_panel_orientation_quirks 17180 1 drm
dca 15130 2 igb,ioatdma
i2c_algo_bit 13413 2 igb,mgag200

Any idea whats going wrong?

Best regards
Frank

[SantoshY_Intel]

Hi,

Thanks for posting in Intel communities.

Could you please provide us with the outputs for the below commands?

1. $ucx_info -d | grep Transport
2. $ibv_devinfo
3. $lspci | grep Mellanox
4. $fi_info --list

Also, please provide us with the below details:

1. Job scheduler
2. CPU details

Thanks & Regards,

Santosh

[Frank_R_1]

$ export UCX_TLS=rc,ud,sm,self
$ ucx_info -d | grep Transport
# Transport: posix
# Transport: sysv
# Transport: self
# Transport: tcp
# Transport: tcp
# Transport: cma

$ ibv_devinfo
hca_id: mlx4_0
transport: InfiniBand (0)
fw_ver: 2.40.7000
node_guid: 0025:90ff:ff17:3624
sys_image_guid: 0025:90ff:ff17:3627
vendor_id: 0x02c9
vendor_part_id: 4099
hw_ver: 0x1
board_id: SM_2181000001000
phys_port_cnt: 1
port: 1
state: PORT_ACTIVE (4)
max_mtu: 4096 (5)
active_mtu: 4096 (5)
sm_lid: 1
port_lid: 3
port_lmc: 0x00
link_layer: InfiniBand

$ lspci | grep Mellanox
06:00.0 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3]

$ fi_info --list
psm2:
version: 1.7
psm:
version: 1.7
usnic:
version: 1.0
ofi_rxm:
version: 1.0
ofi_rxd:
version: 1.0
verbs:
version: 1.0
UDP:
version: 1.1
sockets:
version: 2.0
tcp:
version: 0.1
ofi_perf_hook:
version: 1.0
ofi_noop_hook:
version: 1.0
shm:
version: 1.0
ofi_mrail:
version: 1.0

$ cat /proc/cpuinfo | grep "model name"
model name : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz

Queue System : Torque

[SantoshY_Intel]

Hi,

Thanks for providing the details.

Below are our observations:
1. There is no MLX in your fi_info --list output.

It is likely that the Mellanox OFED (OpenFabrics Enterprise Distribution) drivers may not be installed on your system. Make sure that the Mellanox OFED drivers are installed and configured properly. You can check whether the Mellanox OFED drivers are installed by running the command: "$ ofed_info -s".

2. There is no transport related to mlx from the output of "$ ucx_info -d | grep Transport" command.

Check whether the MLNX_OFED stack is installed on your system, you can use the command: $ rpm -qa | grep mlnx

If the output shows that the MLNX_OFED stack is not installed, you will need to install it first.

it's also possible that your Mellanox network adapter is not supported by UCX or that there is some other configuration issue with your system.

3. Also, I can see that you are using older versions of libfabrics.

According to the "Fabric Software" section of Intel MPI system requirements, the minimum OFI version required is 1.5.0 but, it is recommended to use the latest OFI version. To check your OFI version, use the command: $ fi_info --version

Please update your libfabrics and try again. For more information, please go through Building and Installing Libfabric from the Source article.

Thanks & Regards,

Santosh

[SantoshY_Intel]

Hi,

We haven't heard back from you. Could you please provide us with an update on your issue?

Thanks & Regards,

Santosh

[SantoshY_Intel]

Hi,

I have not heard back from you. This thread will no longer be monitored by Intel. If you need further assistance, please post a new question.

Thanks & Regards,

Santosh