Intel® oneAPI HPC Toolkit
Get help with building, analyzing, optimizing, and scaling high-performance computing (HPC) applications.
1890 Discussions

CPU usage 100% during MPI blocking communication waiting

yinyin
Beginner
722 Views

In my program, according to the user input, the master process will broadcast data to other worker processes to do computation, after one time computation, the worker processes will wait for new data once again and do the corresponding computation.

Worker processes  use a do-while loop to keep receiving data from master like this,

 

int master_rank =0
int mycmd;
if(my_rank==master_rank )
{
  //wait for user input
  //accept input command
  MPI_Bcast(&mycmd, 1, MPI_INT, master_rank, MPI_COMM_WORLD);  ​
  //do some work
}
else
{
  MPI_Bcast(&mycmd, 1, MPI_INT, master_rank, MPI_COMM_WORLD);
  switch (mycmd)
  {
    case  1:
       // do some work
       break;
    case 2:
       // do some work
       break;
    default:
       std::cout << "myrank:" << my_rank << "Unknown command:" << mycmd << std::endl;
       break;

  }
} 

However, even during waiting for user input, the worker processes will keep 100% cpu running, which make  PC too hot.

 

After research in web, I found the following parameters can decrease cpu usage ratio,

I_MPI_WAIT_MODE=0
I_MPI_THREAD_YIELD=3
I_MPI_THREAD_SLEEP=10

 

But this also make computation itself become low performance, that is absolutely not what I want .

 

Does intel MPI library provide some equivalent function to control the MPI behavior during running? just like openmp's   omp_set_xxx function.

 

Thank you for your help. 

0 Kudos
9 Replies
SantoshY_Intel
Moderator
688 Views

Hi,


Thanks for reaching out to us.

We are working on your issue and we will get back to you soon.


Thanks & regards,

Santosh


SantoshY_Intel
Moderator
662 Views

Hi,


Could you please provide us the sample reproducer code along with the commands you used to compile and run?


>>"I_MPI_THREAD_SLEEP=10"

As the sleep time increases, there will be a drop in performance. So try to reduce sleep time in order to enhance the performance.


>>"Does intel MPI library provide some equivalent function to control the MPI behavior during running? just like openmp's  omp_set_xxx function."

Could you please elaborate on what kind of MPI behavior to control?


Thanks,

Santosh


yinyin
Beginner
653 Views

Dear Santosh

 Maybe my description was not clear.

  I want to make an emphasize that

 - I DO want to drop/decrease response performance during waiting procedure  by setting I_MPI_THREAD_SLEEP a big value. 

 - But once the worker processes receive the command(which is broadcast from master process), these worker processes should run  the computing procedure at their full power, where a zero   I_MPI_THREAD_SLEEP is wanted.

I wonder if I can adjust  I_MPI_THREAD_SLEEP during program running.

 

Expecting  your reply.

 

SantoshY_Intel
Moderator
625 Views

Hi,

 

>>"I wonder if I can adjust  I_MPI_THREAD_SLEEP during program running"
We can not adjust I_MPI_THREAD_SLEEP during program running. Please find more information regarding usage of I_MPI_THREAD_SLEEP from the below link:

https://software.intel.com/content/www/us/en/develop/documentation/mpi-developer-reference-linux/top...

 

We need more information to investigate your issue.

So, Could you please provide us the sample reproducer code along with the commands you used to compile and run?"

 

>>"Does intel MPI library provide some equivalent function to control the MPI behavior during running? just like openmp's  omp_set_xxx function."

Could you please elaborate on what kind of MPI behavior to control?

 

Thanks & Regards,

Santosh

 

SantoshY_Intel
Moderator
605 Views

Hi,

We have not heard back from you. Could you please provide the details that have been asked in my previous post?

 

Thanks,

Santosh

 

yinyin
Beginner
599 Views

Dear Santosh

 I haven't solved this. Sorry for my late response.

I will give a minimal program to show this issue in several days.

Please keep tracking my issue.

 

B.R.

YIN

SantoshY_Intel
Moderator
556 Views

Hi,

 

We have not heard back from you. Could you please provide us the sample reproducer code along with the commands you used to compile and run?

 

Thanks,

Santosh

 

 

SantoshY_Intel
Moderator
539 Views

Hi,


>>"I will give a minimal program to show this issue in several days.Please keep tracking my issue."

We have been waiting for your response. But we haven't heard back from you. Can you provide the reproducer code along with the commands you used to encounter the issue.


Thanks & Regards,

Santosh


SantoshY_Intel
Moderator
459 Views

Hi,

 

As we haven't heard back from you, we are closing this thread for now. However, feel free to raise a new query whenever you can provide us a sample reproducer as this thread will no longer be monitored. 

Have a Good day!

 

Thanks,

Santosh

 

Reply