topic Can the same effect be in Software Archive

Multiple MPI task offload to the same MIC.

aketh_t_ — Mon, 21 Mar 2016 06:32:15 GMT

Hi all,

I am currently looking to offload from different MPI processor to the same MIC.

The offloads are asynchronous in nature.

What I would like to know is there a queue implemented by default that makes sure only a given MPI task at a time executes on the MIC?

Is there a partition of the MIC we can carry out such that each task offloads to a subset of cores?

How do I synchronise if no such thing exists?

Whats the solution?

You could use the kmp_place

TimP — Mon, 21 Mar 2016 12:10:43 GMT

You could use the kmp_place_threads environment setting to cause each process to use a distinct group of cores on the mic.

If you are actually using offload mode you would prepend mic_env_prefix, of course.

The processes would still contend for use of mic core 0 for data transfer etc. so you would not assign work to that core.

The processes would still

James_C_Intel2 — Mon, 21 Mar 2016 12:21:00 GMT

The processes would still contend for use of mic core 0 for data transfer etc. so you would not assign work to that core.

The offload library automatically removes the core used by the offload data-transfer engine from the affinity mask of the processes it creates, so the OpenMP runtime won't create threads for it, or try to run anything on it. Therefore this all happens "automagically" and you shouldn't need to do anything special.

Alternatively, if you're not already using OpenMP, look at hstreams, which might be a nicer solution for you.

How do i set a different

aketh_t_ — Mon, 21 Mar 2016 14:55:26 GMT

How do i set a different environment for every task?

if I have 128 tasks how do i do it?

Quote:aketh t. wrote:

Paulius_V_1 — Mon, 21 Mar 2016 15:09:05 GMT

aketh t. wrote:

How do i set a different environment for every task?

if I have 128 tasks how do i do it?

mpirun -configfile configurationfile

where configuration file launches ranks on each on each node. Here you can give a different set of environment variables to each rank. Most importantly - MIC_KMP_PLACE_THREADS=x,x,x this allows you to set the offset on the MIC so your ranks don't all offload to the same cores.

Can the same effect be

aketh_t_ — Tue, 12 Apr 2016 06:23:32 GMT

Can the same effect be achieved with OpenMP.

I.e different OpenMP threads offload to the same MIC but to a subsets of the cores.