Coprocessor binary not used when doing symmetric MPI

[Dan_M_4]

Hello,

I am unable to use Intel MPI symmetric mode accross the host and Xeon Phi because the mpirun command appears to use the same binary for both the processor and the coprocessor. I am using NFS to share the same file system on the host and the MIC card and am following this article: http://software.intel.com/en-us/articles/using-xeon-phi-prefixes-and-extensions-for-intel-mpi-jobs-in-nfs-shared-environment

For example, if I have a small MPI program called hello.c:

$ export I_MPI_MIC=enable

$ export I_MPI_POSTFIX=.MIC

$ mpiicc -o hello hello.c

$ mpiicc -mmic -o hello.MIC hello.c

$ echo $(hostname) > mpi_hosts

$ mpirun -n 2 -f mpi_hosts ./hello

CPU: Hello from processor 1 of 2

CPU: Hello from processor 1 of 0

$ echo $(hostname)-mic0 > mpi_hosts

$ mpirun -n 2 -f mpi_hosts ./hello.MIC

MIC: Hello from processor 1 of 2

MIC: Hello from processor 0 of 2

$ echo $(hostname)-mic0 > mpi_hosts; echo $(hostname) >> mpi_hosts

$ mpirun -perhost 1 -n 2 -f mpi_hosts ./hello

In the final line, I receive the following error messages before the program hangs:

./hello: line 1: 0: not found
./hello: line 1: 4: not found
./hello: line 1: : Permission denied
./hello: line 1: ELF: not found
./hello: line 5: syntax error: unexpected "("

These are the same messages that I receive if I run the CPU binary on the Xeon Phi card:

$ ssh mic0 ~/hello

So, it appears that mpirun is using the same binary on both the CPU and the Xeon Phi, eventhough I have set I_MPI_MIC=enable and I_MPI_POSTFIX=.MIC. I have tried many combinations of I_MPI_POSTFIX and I_MPI_PREFIX, but the result is always the same. They appear to be ignored. I have also tried loading these environment variables in my .bashrc file hoping to set them in new sessions that might get created.

Please let me know if there is something that I am missing. Thanks in advance.

-Dan

[Frances_R_Intel · Accepted solution]

This is one of those things that you stare at and stare at and can't see what is wrong, ... until it finally hits you:

I_MPI_MIC_POSTFIX not I_MPI_POSTFIX

Sorry it took me so long to see this.

[Dan_M_4]

Thank you! I think I got distracted by dubious advice from a non-Intel website. I used the correct environment variables and now it works.

-Dan