Software Archive
Read-only legacy content
17061 Discussions

Multiple MPI task offload to the same MIC.

aketh_t_
Beginner
545 Views

Hi all,

I am currently looking to offload from different MPI processor to the same MIC.

The offloads are asynchronous in nature.

What I would like to know is there a queue implemented by default that makes sure only a given MPI task at a time executes on the MIC?

Is there a partition of the MIC we can carry out such that each task offloads to a subset of cores?

How do I synchronise if no such thing exists?

Whats the solution?

0 Kudos
5 Replies
TimP
Honored Contributor III
545 Views

You could use the kmp_place_threads environment setting to cause each process to use a distinct group of cores on the mic. 

If you are actually using offload mode you would prepend mic_env_prefix, of course.

The processes would still contend for use of mic core 0 for data transfer etc. so you would not assign work to that core.

0 Kudos
James_C_Intel2
Employee
545 Views

The processes would still contend for use of mic core 0 for data transfer etc. so you would not assign work to that core.

The offload library automatically removes the core used by the offload data-transfer engine from the affinity mask of the processes it creates, so the OpenMP runtime won't create threads for it, or try to run anything on it. Therefore this all happens "automagically" and you shouldn't need to do anything special.

Alternatively, if you're not already using OpenMP, look at hstreams, which might be a nicer solution for you.

0 Kudos
aketh_t_
Beginner
545 Views

How do i set a different environment for every task?

if I have 128 tasks how do i do it?

0 Kudos
Paulius_V_1
Beginner
545 Views

aketh t. wrote:

How do i set a different environment for every task?

if I have 128 tasks how do i do it?

 

mpirun -configfile configurationfile

where configuration file launches ranks on each on each node. Here you can give a different set of environment variables to each rank. Most importantly - MIC_KMP_PLACE_THREADS=x,x,x this allows you to set the offset on the MIC so your ranks don't all offload to the same cores. 

 

0 Kudos
aketh_t_
Beginner
545 Views

Can the same effect be achieved with OpenMP.

I.e different OpenMP threads offload to the same MIC but to a subsets of the cores.

0 Kudos
Reply