intel mpi program starts on proc0

[yimwlihpc_a-star_edu]

Dear all,

I am very new to try intel mpi, which I found very powerful and obtained performance gain relative to generic lam/mpich.

I can access to a node which has 16 cpus. We run the jobs interactively on this node. In the presence of some serial jobs, I started my application and found that there is always one process sharing cpu power with one of the serial program. The top message looks like:
-----------------------------------------------------------------------------
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4896USER2 20 0 24.5g 79m 2312 R 100 0.1 30:55.20 seward.exe
5325 wlyim 20 0 999m 943m 9.9m R 100 1.5 0:27.82 cp2k.popt
5326 wlyim 20 0 1037m 940m 9m R 100 1.5 0:27.62 cp2k.popt
5327 wlyim 20 0 1037m 939m 9.9m R 100 1.5 0:27.32 cp2k.popt
5226USER2 20 0 24.5g 37m 1884 R 50 0.1 4:04.88 seward.exe
5328 wlyim 20 0 987m 939m 10m R 50 1.5 0:14.52 cp2k.popt
---------------------------------------------------------------------------------

I run my job using >>> mpirun -f ./hosts -n 4 ./cp2k.popt -i ...

Can anybody point out what's wrong with my mpi command? I guess there may be something to avoid proc0 but I don't know what command arguement does this job.

Many thanks in advance!

Best regards,
William

[TimP]

If you wish to stop MPI from using certain cores, you will have to set the environment variable I_MPI_PIN_PROCS accordingly. The options would be described in the documents. You would have to take equivalent action for other jobs, such as taskset, to restrict them to certain cores. There will still be competition for resources, including cache and memory access, which will likely prevent either application from reaching full performance.

[yimwlihpc_a-star_edu]

Quoting - tim18

If you wish to stop MPI from using certain cores, you will have to set the environment variable I_MPI_PIN_PROCS accordingly. The options would be described in the documents. You would have to take equivalent action for other jobs, such as taskset, to restrict them to certain cores. There will still be competition for resources, including cache and memory access, which will likely prevent either application from reaching full performance.

Hi tim18,

I am very appreciated for your quick reply! And thanks for guiding me to a probable solution.

Best regards,
William

[yimwlihpc_a-star_edu]

Quoting - yimwlihpc.a-star.edu.sg

Hi tim18,

I am very appreciated for your quick reply! And thanks for guiding me to a probable solution.

Best regards,
William

Following tim18's instruction, it succeeded. But as tim18 said, all other jobs have to be dedicated to a particular core, e.g. by taskset. In my case, the serial job link-process would change proc and I didn't ask my colleague to kill his job to test.

Alternatively, I got another workaround by simply setting the following lines.

-----------------------------------
export I_MPI_DEVICE=shm
export I_MPI_PIN_PROCS=
export I_MPI_DEVICE=shm
----------------------------------

Best regards,
William