Any luck with OFED-3.5-1-MIC-beta1 on CentOS 6.5?

[Andrey_Vladimirov]

We are trying to run with an unmodified kernel of 6.5 and MPSS 3.2.1. Here is what happens:

[bash]

[root@c001-n004 OFED-3.5-1-MIC-beta1]# uname -a
Linux c001-n004 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
[root@c001-n004 OFED-3.5-1-MIC-beta1]#
[root@c001-n004 OFED-3.5-1-MIC-beta1]# lspci | grep Mellanox
01:00.0 InfiniBand: Mellanox Technologies MT27600 [Connect-IB]
[root@c001-n004 OFED-3.5-1-MIC-beta1]# ./install.pl -c ofed.conf -n ofed_net.conf
...
Running rpm -iv  /root/OFED-3.5-1-MIC-beta1/RPMS/centos-release-6-5.el6.centos.11.1/x86_64/ofed-docs-3.5-OFED.3.5.1.x86_64.rpm
Install intel-mic-psm RPM:
Running rpm -iv  --nodeps /root/OFED-3.5-1-MIC-beta1/RPMS/centos-release-6-5.el6.centos.11.1/x86_64/intel-mic-psm-3.1-1.x86_64.rpm
Install intel-mic-psm-devel RPM:
Running rpm -iv  --nodeps /root/OFED-3.5-1-MIC-beta1/RPMS/centos-release-6-5.el6.centos.11.1/x86_64/intel-mic-psm-devel-3.1-1.x86_64.rpm
Device (15b3:1011):
    01:00.0 InfiniBand: Mellanox Technologies MT27600 [Connect-IB]
    Link Width: 8x
    PCI Link Speed: Unknown

Installation finished successfully.

[/bash]

and then...

[bash]

[root@c001-n004 OFED-3.5-1-MIC-beta1]# service openibd start
Loading Mellanox HCA driver:                               [FAILED]
Loading Mellanox MLX4 HCA driver:                          [FAILED]
Loading Mellanox MLX4_IB HCA driver:                       [FAILED]
Loading Mellanox MLX4_EN HCA driver:                       [FAILED]
Loading Mellanox MLX5 HCA driver:                          [FAILED]
Loading Mellanox MLX5_IB HCA driver:                       [FAILED]
Loading cxgb3 driver:                                      [FAILED]
Loading cxgb4 driver:                                      [FAILED]
Loading nes driver:                                        [FAILED]
Loading HCA driver and Access Layer:                       [FAILED]

Please open an issue in the http://bugs.openfabrics.org and attach /tmp/ib_debug_info.log

[root@c001-n004 OFED-3.5-1-MIC-beta1]# ibv_devinfo
Failed to get IB devices list: Function not implemented
[root@c001-n004 OFED-3.5-1-MIC-beta1]# ibstat
ibwarn: [45515] umad_init: can't read ABI version from /sys/class/infiniband_mad/abi_version (No such file or directory): is ib_umad module loaded?
ibpanic: [45515] main: can't init UMAD library: No such file or directory
[root@c001-n004 OFED-3.5-1-MIC-beta1]# ibstatus
Fatal error:  No devices
[root@c001-n004 OFED-3.5-1-MIC-beta1]#
[/bash]

Has anyone seen this?

[TaylorIoTKidd]

Andrey,

A similar problem has shown up before (https://software.intel.com/en-us/forums/topic/471710).

There have been issues with previous vendor specific OFED installs causing problems, for example, for some reason one or more of the vendor rpms is not being replaced.

Try the following:

(1) Uninstall the previous OFED

(2) Install the generic OFED

Regards
--
Taylor
 

[Tommi_T_]

You have to apply patch from this bug. It's clear sign that no one at Intel has tested that combination at all...

http://bugs.openfabrics.org/bugzilla/show_bug.cgi?id=2466

[TaylorIoTKidd]

Tommi,

Thanks. I will pass the information up.

Regards
--
Taylor
 

[Tommi_T_ · Accepted solution]

New OFED-3.5-2-MIC-beta1 has been released which should fix this issue:

http://lists.openfabrics.org/pipermail/ewg/2014-May/018129.html

[Andrey_Vladimirov]

Terrific! OFED-3.5-2-MIC-beta1 solved my problem. Thank you, Tommi!