dev open failed for mlx4_0, err=Permission denied

[Miguel_M_]

Hi, 

I have this error with the Infiniband inizialitation when I launch a program with Intel MPI (with OpenMPI as well):

$ mpirun ./testC.exe
[-1] MPI startup(): Imported environment partly inaccesible. Map=0 Info=15c9ff0
[0] DAPL startup(): trying to open default DAPL provider from dat registry: ofa-v2-mlx4_0-1
node2.************:286f:14713700: 1877 us(1877 us): open_hca: dev open failed for mlx4_0, err=Permission denied

(...)

When it worked it was like this:

$ mpirun ./testC.exe

[-1] MPI startup(): Imported environment partly inaccesible. Map=0 Info=1c29040

[0] DAPL startup(): trying to open default DAPL provider from dat registry: ofa-v2-mlx4_0-1

[0] MPI startup(): DAPL provider ofa-v2-mlx4_0-1

(...)

I have been searching this forum and googling a lot but I haven't been able to find this same problem. I don't know what this error means.

I have to say that this error is apearing in two different clusters. In the first one it started to happen suddenly. In the second it started to happen after we changed our Mellanox switch for a new one identical to the previous. So in both cases configuration files and all software remain unchanged.

What could be the cause for this?

Regards, 

Miguel

[James_T_Intel]

Hi Miguel, I'm checking with our DAPL developer for more information on this. Sincerely, James Tullos Technical Consulting Engineer Intel® Cluster Tools

[Miguel_M_]

Hi James, Thank you. I think I know now where the error is. It seem that permissions for devices in /dev/infiniband are not correctly set. Regards, Miguel

[James_T_Intel]

Hi Miguel, Great! I'm glad to hear it's sorted out. Sincerely, James Tullos Technical Consulting Engineer Intel® Cluster Tools