https://community.intel.com/t5/Software-Archive/Multi-Mic-affinity/m-p/1062191#M54545 <P>I would like to run a heterogeneous program on a multi node cluster where each node comes with 2 MIC devices.<BR /> Let's consider for example 2 dual-sockets nodes:</P> <P>------------------- &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -------------------<BR /> --- NODE 1 --&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; --- NODE 2 --<BR /> &nbsp;&nbsp;&nbsp;&nbsp; &nbsp; |&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp;&nbsp; &nbsp; &nbsp; |<BR /> &nbsp;S&nbsp;&nbsp;&nbsp; |&nbsp; S&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp;&nbsp;&nbsp; S&nbsp;&nbsp;&nbsp; |&nbsp; S&nbsp; &nbsp;<BR /> &nbsp;o&nbsp;&nbsp; &nbsp; |&nbsp; o&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; o&nbsp;&nbsp;&nbsp; |&nbsp; o&nbsp; &nbsp;<BR /> &nbsp;c&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp; c&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; c&nbsp;&nbsp;&nbsp; |&nbsp; c&nbsp; &nbsp;<BR /> &nbsp;k&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp; k&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; k&nbsp;&nbsp;&nbsp; |&nbsp; k&nbsp; &nbsp;<BR /> &nbsp;e&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp; e&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; e&nbsp;&nbsp;&nbsp; |&nbsp; e&nbsp; &nbsp;<BR /> &nbsp;t&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp; t&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; t&nbsp;&nbsp;&nbsp; |&nbsp; t&nbsp; &nbsp;<BR /> &nbsp;#1&nbsp;&nbsp;&nbsp; |&nbsp; #2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp; #1&nbsp;&nbsp; |&nbsp; #2 &nbsp;<BR /> -------------------&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -------------------<BR /> MIC0&nbsp; | MIC1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; MIC0&nbsp; | MIC1<BR /> -------------------&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -------------------</P> <P>If I execute 4 mpi processes, how can I assure that each mpi process will run on a different socket, and that each mpi-rank will offload on the MIC device connected to the socket where it is executed?</P> Sun, 07 Feb 2016 00:30:52 GMT ale3 2016-02-07T00:30:52Z https://community.intel.com/t5/Software-Archive/Multi-Mic-affinity/m-p/1062191#M54545 <P>I would like to run a heterogeneous program on a multi node cluster where each node comes with 2 MIC devices.<BR /> Let's consider for example 2 dual-sockets nodes:</P> <P>------------------- &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -------------------<BR /> --- NODE 1 --&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; --- NODE 2 --<BR /> &nbsp;&nbsp;&nbsp;&nbsp; &nbsp; |&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp;&nbsp; &nbsp; &nbsp; |<BR /> &nbsp;S&nbsp;&nbsp;&nbsp; |&nbsp; S&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp;&nbsp;&nbsp; S&nbsp;&nbsp;&nbsp; |&nbsp; S&nbsp; &nbsp;<BR /> &nbsp;o&nbsp;&nbsp; &nbsp; |&nbsp; o&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; o&nbsp;&nbsp;&nbsp; |&nbsp; o&nbsp; &nbsp;<BR /> &nbsp;c&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp; c&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; c&nbsp;&nbsp;&nbsp; |&nbsp; c&nbsp; &nbsp;<BR /> &nbsp;k&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp; k&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; k&nbsp;&nbsp;&nbsp; |&nbsp; k&nbsp; &nbsp;<BR /> &nbsp;e&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp; e&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; e&nbsp;&nbsp;&nbsp; |&nbsp; e&nbsp; &nbsp;<BR /> &nbsp;t&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp; t&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; t&nbsp;&nbsp;&nbsp; |&nbsp; t&nbsp; &nbsp;<BR /> &nbsp;#1&nbsp;&nbsp;&nbsp; |&nbsp; #2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp; #1&nbsp;&nbsp; |&nbsp; #2 &nbsp;<BR /> -------------------&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -------------------<BR /> MIC0&nbsp; | MIC1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; MIC0&nbsp; | MIC1<BR /> -------------------&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -------------------</P> <P>If I execute 4 mpi processes, how can I assure that each mpi process will run on a different socket, and that each mpi-rank will offload on the MIC device connected to the socket where it is executed?</P> Sun, 07 Feb 2016 00:30:52 GMT https://community.intel.com/t5/Software-Archive/Multi-Mic-affinity/m-p/1062191#M54545 ale3 2016-02-07T00:30:52Z https://community.intel.com/t5/Software-Archive/Multi-Mic-affinity/m-p/1062192#M54546 <P>Hi Ale,</P> <P>Did you try using &nbsp;"mpirun -ppn 2 -np 4 I_MPI_PIN_DOMAIN=socket ....".</P> <P>Also I am assuming that you have Intel Xeon Phi coprocessors configured in external bridge topology and are able to ping each other across cluster.</P> <P>Thanks</P> Thu, 11 Feb 2016 21:43:44 GMT https://community.intel.com/t5/Software-Archive/Multi-Mic-affinity/m-p/1062192#M54546 Sunny_G_Intel 2016-02-11T21:43:44Z