ib_read_bw not working on MIC

[Fedele_S_]

Hello,

this is my configuration:
CentOS 6.6, IntelIB 7.4.0.0.21
on host:
server# ibv_devinfo
hca_id:    scif0
    transport:            iWARP (1)
    fw_ver:                0.0.1
    node_guid:            4c79:baff:fe66:0773
    sys_image_guid:            4c79:baff:fe66:0773
    vendor_id:            0x8086
    vendor_part_id:            0
    hw_ver:                0x1
    phys_port_cnt:            1
        port:    1
            state:            PORT_ACTIVE (4)
            max_mtu:        4096 (5)
            active_mtu:        4096 (5)
            sm_lid:            1
            port_lid:        1000
            port_lmc:        0x00
            link_layer:        Ethernet

hca_id:    qib0
    transport:            InfiniBand (0)
    fw_ver:                0.0.0
    node_guid:            0011:7500:006f:76dc
    sys_image_guid:            0011:7500:006f:76dc
    vendor_id:            0x1175
    vendor_part_id:            29474
    hw_ver:                0x2
    board_id:            InfiniPath_QLE7340
    phys_port_cnt:            1
        port:    1
            state:            PORT_ACTIVE (4)
            max_mtu:        4096 (5)
            active_mtu:        2048 (4)
            sm_lid:            1
            port_lid:        26
            port_lmc:        0x00
            link_layer:        InfiniBand

on device:

server-mic# ibv_devinfo
hca_id:    scif0
    transport:            SCIF (2)
    fw_ver:                0.0.1
    node_guid:            4c79:baff:fe66:0772
    sys_image_guid:            4c79:baff:fe66:0772
    vendor_id:            0x8086
    vendor_part_id:            0
    hw_ver:                0x1
    phys_port_cnt:            1
        port:    1
            state:            PORT_ACTIVE (4)
            max_mtu:        4096 (5)
            active_mtu:        4096 (5)
            sm_lid:            1
            port_lid:        1001
            port_lmc:        0x00
            link_layer:        SCIF

# ib_read_bw mic0
---------------------------------------------------------------------------------------
Device not recognized to implement inline feature. Disabling it
---------------------------------------------------------------------------------------
                    RDMA_Read BW Test
 Dual-port       : OFF        Device         : scif0
 Number of qps   : 1        Transport type : IW
 Connection type : RC        Using SRQ      : OFF
 TX depth        : 128
 CQ Moderation   : 100
 Mtu             : 4096
 Link type       : Ethernet
 Gid index       : 0
 Outstand reads  : 255
 rdma_cm QPs     : OFF
 Data ex. method : Ethernet
---------------------------------------------------------------------------------------
 local address: LID 0x3e8 QPN 0x0003 PSN 0xbaa278 OUT 0xff RKey 0x000002 VAddr 0x002b75ebc6f000
 GID: 76:121:186:102:07:115:00:00:00:00:00:00:00:00:00:00
ethernet_read_keys: Couldn't read remote address
 Unable to read from socket/rdam_cm
Failed to exchange data between server and clients

[Fedele_S_]

If i run on server

ib_read_bw -d qib0 mic0

the output is:

---------------------------------------------------------------------------------------
Device not recognized to implement inline feature. Disabling it
---------------------------------------------------------------------------------------
                    RDMA_Read BW Test
 Dual-port       : OFF        Device         : qib0
 Number of qps   : 1        Transport type : IB
 Connection type : RC        Using SRQ      : OFF
 TX depth        : 128
 CQ Moderation   : 100
 Mtu             : 2048
 Link type       : IB
 Outstand reads  : 16
 rdma_cm QPs     : OFF
 Data ex. method : Ethernet
---------------------------------------------------------------------------------------
 local address: LID 0x1a QPN 0x556507 PSN 0x590737 OUT 0x10 RKey 0x41e8cf00 VAddr 0x002b0dc2778000
 remote address: LID 0x3e9 QPN 0x0002 PSN 0xb20850 OUT 0xff RKey 0x000001 VAddr 0x007f4dcc4ce000
---------------------------------------------------------------------------------------
 #bytes     #iterations    BW peak[MB/sec]    BW average[MB/sec]   MsgRate[Mpps]
Problems with warm up

by the way if this can help:

# ibscif-opt
mic0:
  blocking_recv = 1
  blocking_send = 0
  check_grh = 1
  fast_rdma = 1
  host_proxy = 0
  max_pinned = 50
  new_ib_type = 1
  rma_threshold = 1024
  scif_loopback = 1
  verbose = 0
  window_size = 40

host:
  blocking_recv = 1
  blocking_send = 0
  check_grh = 1
  fast_rdma = 1
  host_proxy = 0
  max_pinned = 50
  new_ib_type = 0
  rma_threshold = 1024
  scif_loopback = 1
  verbose = 0
  window_size = 40

[Patrick_M_Intel2]

It looks like you are trying to use "qib-direct" from the card: where the host's qib card is made to appear as native on the mic card. However that does not appear to be working: ibv_devinfo on the mic does not show any qib devices on the card.