Can't get information about Omnipath HFI on RHEL 7.3 hosts

[JBlaa]

Recently we got some new KNL nodes and decided to try RHEL 7.3 on these hosts running the 3.10.0-514.10.2.el7.x86_64 kernel.

After installing the IntelOPA-Basic software and upgrading the firmware on the HFI and rebooting the nodes, we still can't get anything other than the following from opainfo

[root@sknl0701 ~]# opainfo

oib_utils ERROR: [7534] open_verbs_ctx: failed to find verbs device

opainfo: Unable to open hfi:port 0:1

Even though the software and firmware never complains about any errors we can still see that even after forcing dracut to recreate the system image the hfi1 driver will not load.

[root@shas0101 ~]# lsmod | grep hfi1

hfi1 633634 1

rdmavt 57992 1 hfi1

ib_mad 47817 5 hfi1,ib_cm,ib_sa,rdmavt,ib_umad

ib_core 98787 14 hfi1,rdma_cm,ib_cm,ib_sa,iw_cm,xprtrdma,ib_mad,ib_ucm,rdmavt,ib_iser,ib_umad,ib_uverbs,ib_ipoib,ib_isert

i2c_algo_bit 13413 2 hfi1,mgag200

i2c_core 40582 6 drm,hfi1,ipmi_ssif,drm_kms_helper,mgag200,i2c_algo_bit

[root@sknl0701 ~]# modprobe -v hfi1

[root@sknl0701 ~]# lsmod | grep hfi1

hfi1 697628 0

rdmavt 63294 1 hfi1

ib_core 210381 13 hfi1,rdma_cm,ib_cm,iw_cm,rpcrdma,ib_ucm,rdmavt,ib_iser,ib_umad,ib_uverbs,rdma_ucm,ib_ipoib,ib_isert

i2c_algo_bit 13413 3 igb,hfi1,mgag200

i2c_core 40756 7 drm,igb,hfi1,ipmi_ssif,drm_kms_helper,mgag200,i2c_algo_bit

[root@sknl0701 ~]# yum info libibmad

Loaded plugins: product-id, search-disabled-repos, subscription-manager

Installed Packages

Name : libibmad

Arch : x86_64

Version : 1.3.12

Release : 1.el7

Size : 132 k

Repo : installed

From repo : curc

Summary : OpenFabrics Alliance InfiniBand MAD library

URL : http://openfabrics.org/

License : GPLv2 or BSD

Description : libibmad provides low layer IB functions for use by the IB diagnostic

: and management programs. These include MAD, SA, SMP, and other basic

: IB functions.

[root@sknl0701 ~]# yum info libibmad-devel

Loaded plugins: product-id, search-disabled-repos, subscription-manager

<spa...

[idata]

Hello @Wilshire461,

I am sure our Intel Omni-Path technical support team can assist you with this, however, they do not provide support through this venue.I am forwarding your information for them to contact you directly. Have a nice day!Dave A.

[idata]

I thought you might also want to have this information handy.

http://www.intel.com/content/www/us/en/support/network-and-i-o/fabric-products/000007466.html Support for Intel(R) Fabric ProductsDave A.