Intel® oneAPI HPC Toolkit
Get help with building, analyzing, optimizing, and scaling high-performance computing (HPC) applications.
Announcements
This community is designed for sharing of public information. Please do not share Intel or third-party confidential information here.

Problems with 512 cores or more.

Adalberto_Fazzio
Beginner
92 Views
Hello all!
When I try to run an application with 512 cores (or more) I get this MPI error:
[33] MPI startup(): ofa fabric is not available and fallback fabric is not enabled
I'm usingI_MPI_FABRICS=shm:ofa, but with DAPL I also get an MPI error.
Using 256 cores or less I got no errors, it works like a charm.
The system is a SGI cluster with quad-socket AMD Interlagos nodes (64 cores per node). Each node as two Infiniband ports and theI_MPI_OFA_NUM_PORTS=2 is enabled.
Thank you in advance!
0 Kudos
1 Reply
Dmitry_K_Intel2
Employee
92 Views
Hi Adalberto,

To answer your question we need to get more information.
Could you please provide the command line and environment variables you set before mpirun.
Add "-env I_MPI_DEBUG 6" to the command line and send us the output (you can send a private message if don't want to share your information with all readers - see option "mark this post private" below).

Regards!
Dmitry
Reply