RDMA troubles after install MIC

Rachko__Anton · ‎06-20-2016

After install MIC on compute node, i had problem with MPI.

I use CentOS 6.7 + OFED 3.18.1 with MIC support.

With same options in /etc/modprobe.d/ibscif.conf, i had:

-if options ibscif new_ib_type=1:

hca_id: scif0
transport: invalid transport (-1)

-if options ibscif new_ib_type=0, transport is OK, but in both cases don't work StarCCM+, because it try to use scif as RDMA (because as first RDMA device set scif, "starccm+: Rank 0:6: MPI_Init: rdma_accept() failed").

I set options ibscif enable_fast_rdma=1, it's remove scif device from RDMA devices list and StarCCM+ works.

But I don't understand - this "fast RDMA" mode is good for MIC's?

Loc_N_Intel · ‎06-25-2016

Hi Anton,

What MPSS version did you install? Also, what version of the Intel MPI that you were using?

Thanks.