Intel® MPI Library
Get help with building, analyzing, optimizing, and scaling high-performance computing (HPC) applications.
2154 Discussions

CPU usage 100% during MPI blocking communication waiting

yinyin
Beginner
1,995 Views

In my program, according to the user input, the master process will broadcast data to other worker processes to do computation, after one time computation, the worker processes will wait for new data once again and do the corresponding computation.

Worker processes  use a do-while loop to keep receiving data from master like this,

 

int master_rank =0
int mycmd;
if(my_rank==master_rank )
{
  //wait for user input
  //accept input command
  MPI_Bcast(&mycmd, 1, MPI_INT, master_rank, MPI_COMM_WORLD);  ​
  //do some work
}
else
{
  MPI_Bcast(&mycmd, 1, MPI_INT, master_rank, MPI_COMM_WORLD);
  switch (mycmd)
  {
    case  1:
       // do some work
       break;
    case 2:
       // do some work
       break;
    default:
       std::cout << "myrank:" << my_rank << "Unknown command:" << mycmd << std::endl;
       break;

  }
} 

However, even during waiting for user input, the worker processes will keep 100% cpu running, which make  PC too hot.

 

After research in web, I found the following parameters can decrease cpu usage ratio,

I_MPI_WAIT_MODE=0
I_MPI_THREAD_YIELD=3
I_MPI_THREAD_SLEEP=10

 

But this also make computation itself become low performance, that is absolutely not what I want .

 

Does intel MPI library provide some equivalent function to control the MPI behavior during running? just like openmp's   omp_set_xxx function.

 

Thank you for your help. 

0 Kudos
9 Replies
SantoshY_Intel
Moderator
1,961 Views

Hi,


Thanks for reaching out to us.

We are working on your issue and we will get back to you soon.


Thanks & regards,

Santosh


0 Kudos
SantoshY_Intel
Moderator
1,935 Views

Hi,


Could you please provide us the sample reproducer code along with the commands you used to compile and run?


>>"I_MPI_THREAD_SLEEP=10"

As the sleep time increases, there will be a drop in performance. So try to reduce sleep time in order to enhance the performance.


>>"Does intel MPI library provide some equivalent function to control the MPI behavior during running? just like openmp's  omp_set_xxx function."

Could you please elaborate on what kind of MPI behavior to control?


Thanks,

Santosh


0 Kudos
yinyin
Beginner
1,926 Views

Dear Santosh

 Maybe my description was not clear.

  I want to make an emphasize that

 - I DO want to drop/decrease response performance during waiting procedure  by setting I_MPI_THREAD_SLEEP a big value. 

 - But once the worker processes receive the command(which is broadcast from master process), these worker processes should run  the computing procedure at their full power, where a zero   I_MPI_THREAD_SLEEP is wanted.

I wonder if I can adjust  I_MPI_THREAD_SLEEP during program running.

 

Expecting  your reply.

 

0 Kudos
SantoshY_Intel
Moderator
1,898 Views

Hi,

 

>>"I wonder if I can adjust  I_MPI_THREAD_SLEEP during program running"
We can not adjust I_MPI_THREAD_SLEEP during program running. Please find more information regarding usage of I_MPI_THREAD_SLEEP from the below link:

https://software.intel.com/content/www/us/en/develop/documentation/mpi-developer-reference-linux/top/environment-variable-reference/other-environment-variables.html#other-environment-variables_GUID-A0217433-9A3A-42BC-A27E-C5F555026F23

 

We need more information to investigate your issue.

So, Could you please provide us the sample reproducer code along with the commands you used to compile and run?"

 

>>"Does intel MPI library provide some equivalent function to control the MPI behavior during running? just like openmp's  omp_set_xxx function."

Could you please elaborate on what kind of MPI behavior to control?

 

Thanks & Regards,

Santosh

 

0 Kudos
SantoshY_Intel
Moderator
1,878 Views

Hi,

We have not heard back from you. Could you please provide the details that have been asked in my previous post?

 

Thanks,

Santosh

 

0 Kudos
yinyin
Beginner
1,872 Views

Dear Santosh

 I haven't solved this. Sorry for my late response.

I will give a minimal program to show this issue in several days.

Please keep tracking my issue.

 

B.R.

YIN

0 Kudos
SantoshY_Intel
Moderator
1,829 Views

Hi,

 

We have not heard back from you. Could you please provide us the sample reproducer code along with the commands you used to compile and run?

 

Thanks,

Santosh

 

 

0 Kudos
SantoshY_Intel
Moderator
1,812 Views

Hi,


>>"I will give a minimal program to show this issue in several days.Please keep tracking my issue."

We have been waiting for your response. But we haven't heard back from you. Can you provide the reproducer code along with the commands you used to encounter the issue.


Thanks & Regards,

Santosh


0 Kudos
SantoshY_Intel
Moderator
1,732 Views

Hi,

 

As we haven't heard back from you, we are closing this thread for now. However, feel free to raise a new query whenever you can provide us a sample reproducer as this thread will no longer be monitored. 

Have a Good day!

 

Thanks,

Santosh

 

0 Kudos
Reply