- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi all,
I am currently looking to offload from different MPI processor to the same MIC.
The offloads are asynchronous in nature.
What I would like to know is there a queue implemented by default that makes sure only a given MPI task at a time executes on the MIC?
Is there a partition of the MIC we can carry out such that each task offloads to a subset of cores?
How do I synchronise if no such thing exists?
Whats the solution?
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You could use the kmp_place_threads environment setting to cause each process to use a distinct group of cores on the mic.
If you are actually using offload mode you would prepend mic_env_prefix, of course.
The processes would still contend for use of mic core 0 for data transfer etc. so you would not assign work to that core.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The processes would still contend for use of mic core 0 for data transfer etc. so you would not assign work to that core.
The offload library automatically removes the core used by the offload data-transfer engine from the affinity mask of the processes it creates, so the OpenMP runtime won't create threads for it, or try to run anything on it. Therefore this all happens "automagically" and you shouldn't need to do anything special.
Alternatively, if you're not already using OpenMP, look at hstreams, which might be a nicer solution for you.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
How do i set a different environment for every task?
if I have 128 tasks how do i do it?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
aketh t. wrote:
How do i set a different environment for every task?
if I have 128 tasks how do i do it?
mpirun -configfile configurationfile
where configuration file launches ranks on each on each node. Here you can give a different set of environment variables to each rank. Most importantly - MIC_KMP_PLACE_THREADS=x,x,x this allows you to set the offset on the MIC so your ranks don't all offload to the same cores.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Can the same effect be achieved with OpenMP.
I.e different OpenMP threads offload to the same MIC but to a subsets of the cores.

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page