RDMA problems in gluster on host with MIC

[Fedele_S_]

Hi,
after I have configured ibscif virtual connector for Infiniband  the RDMA gluster transport does not works anymore.

I have CentOS 6.6 and OFED 3.12-1 MPSS 3.5.2 and I have followed the installation instructions of MPSS_Users_Guide (Revision 3.5) that suggested to remove
compat-rdma-devel and compat-rdma packages.

I have noted that running the command:
ib_send_bw
gives the following error:

# ib_send_bw

************************************
* Waiting for client to connect... *
************************************
---------------------------------------------------------------------------------------
                    Send BW Test
 Dual-port       : OFF        Device         : scif0
 Number of qps   : 1        Transport type : IW
 Connection type : RC        Using SRQ      : OFF
 RX depth        : 512
 CQ Moderation   : 100
 Mtu             : 2048
 Link type       : Ethernet
 Gid index       : 0
 Max inline data : 0
 rdma_cm QPs     : OFF
 Data ex. method : Ethernet
---------------------------------------------------------------------------------------
 local address: LID 0x3e8 QPN 0x0003 PSN 0x123123
 GID: 76:121:186:102:03:119:00:00:00:00:00:00:00:00:00:00
ethernet_read_keys: Couldn't read remote address
 Unable to read to socket/rdam_cm
Failed to exchange data between server and clients

Instead using the output of the command

ib_send_bw -d qib0

gives correct results:

# ib_send_bw -d qib0

************************************
* Waiting for client to connect... *
************************************
---------------------------------------------------------------------------------------
                    Send BW Test
 Dual-port       : OFF        Device         : qib0
 Number of qps   : 1        Transport type : IB
 Connection type : RC        Using SRQ      : OFF
 RX depth        : 512
 CQ Moderation   : 100
 Mtu             : 2048
 Link type       : IB
 Max inline data : 0
 rdma_cm QPs     : OFF
 Data ex. method : Ethernet
---------------------------------------------------------------------------------------
 local address: LID 0x0a QPN 0x0169 PSN 0xe0b768
 remote address: LID 0x20 QPN 0x28b280 PSN 0xc3008c
---------------------------------------------------------------------------------------
 #bytes     #iterations    BW peak[MB/sec]    BW average[MB/sec]   MsgRate[Mpps]
 65536      1000             0.00               2160.87           0.034574
---------------------------------------------------------------------------------------