OMP dynamic scheduling on macOS is broken ?

[MGRAV]

Hi everyone,

I have a code that was perfectly working for years (at least 5 years), and still work fine on Linux.

But since I update to the new OneAPI (icpc (ICC) 2021.5.0 20211109), I have the feeling that my "schedule(dynamic,1)" is ignored and that the compiler goes for static ?

Do you have any idea what the origin of this different behavior, and how I could patch it ?

Best,

Mathieu

[MadhuK_Intel]

Hi,

Thank you for posting in Intel communities.

We tried the dynamic scheduling with #pragma omp for to reproduce your issue at our end, and it is working fine not ignoring the schedule(dynamic,1).

We are attaching the code with its output screenshots, which we have executed on macOS. Please go through them and confirm if the schedule use case is correct in the given code.

#include<omp.h>
#include<stdio.h>
int main()
{
        omp_set_num_threads(4);
#pragma omp parallel
{
#pragma omp for schedule(dynamic,1)        
 //#pragma omp for schedule(static)
                for (int i = 0; i < 32; i++)
                {
                        printf("Iteration %d handled by thread %d\n", i, omp_get_thread_num());
                }

}
        return 0;
}

Output for Static scheduling:

Output for Dynamic scheduling:

macOS vesion:

Could you please provide the sample reproducer code and steps to reproduce at our end to investigate your issue further?

Please provide your macOS version and platform details.

Thanks & Regards,

Madhu

[MGRAV]

Hi Madhu,

I can not see your code and result.

The code I used is this one https://github.com/GAIA-UNIL/G2S

However, I did some extra test this morning, and localise the origine of the issue (clearly outside of my code).

using this code 

#include <iostream>
#include <thread>
#include <omp.h>

int main(int argc, char const *argv[]) {

	#pragma omp parallel for num_threads(5) schedule(dynamic,1)
	for (int i = 0; i < 1000; ++i)
	{
		fprintf(stderr, "%d using thread id: %d\n", i, omp_get_thread_num());
		std::this_thread::sleep_for(std::chrono::microseconds(1000));
	}	
	return 0;
}

it's clearly the use of schedule(dynamic,1) that is at the origin of the issue.

I use the command line interface, and compile with

icpc testIcpc.cpp -qopenmp -o test && ./test

I check if my accident I would not have a weird environment variable set or something like that, but for the moment I discovered nothing on this side.

I use a macOS 12.2.1 (21D62), MacBook Pro (15-inch, 2018, (MacBookPro15,3)) using a 2.9 GHz Intel Core i9 6 cœurs.

Let me know if you need more details.

Best,

Mathieu

[MGRAV]

I double check with your code, and clearly on my side it's wrong  

[MadhuK_Intel]

Hi,

>>"I can not see your code and result."

Sorry for the inconvenience caused.

>>"I double check with your code, and clearly on my side it's wrong"

Thank you for the confirmation.

We tried to reproduce your issue with the code you have provided but, we are getting the dynamic behavior. We are attaching the screenshot of your code output. If you are unable to see the screenshots please refresh your browser tab after a while.

Sorry for the inconvenience. We can see from the below URL that you are using an unsupported OS version ( macOS 12.2.1). We recommend to you use any of the supported OS versions mentioned in the given URL.

https://www.intel.com/content/www/us/en/developer/articles/system-requirements/oneapi-c-compiler-system-requirements.html

Thanks and Regards,

Madhu

[MGRAV]

I don't know, but I think it's give an issue.

With a millisecond pause, the number should be printed in the order. 

[MadhuK_Intel]

Hi,

>>"With a millisecond pause, the number should be printed in the order."

We just want to know that is the problem comes only when there is a millisecond and more delay?

>>"I double check with your code, and clearly on my side it's wrong"

There is no millisecond delay in our code. Have you tried the exact code which we have provided? If yes, you are getting static behavior even if there is no delay. Can we assume like this?

The code which we have attached below is doing some different work for even and odd iteration numbers (which might not take the same time delay to execute). This code exhibits the dynamic behavior at our end. Could you please try to run on your platform and confirm with us (what are you getting? static or dynamic)?

Code:

#include <iostream>
#include <thread>
#include <omp.h>

void odd(void)
{
for(int i=0; i<10000;i++);
}

void even(void)
{
for(int i=0; i<20000;i++);
}

int main(int argc, char const *argv[])
{
    #pragma omp parallel for num_threads(5) schedule(dynamic,1)
    for (int i = 0; i < 50; ++i)
    {
        fprintf(stderr, "%d using thread id: %d\n", i, omp_get_thread_num());
    //   std::this_thread::sleep_for(std::chrono::microseconds(200));

        if(i%2==0)
        even();
        else
        odd();
    }
    return 0;
}

output screenshot:

Best regards,

Madhu

[mgravey]

Hi Madhu,

>>>>"With a millisecond pause, the number should be printed in the order."

>>We just want to know that is the problem comes only when there is a millisecond and more delay?

I just mean that the output should be in the order of the for-loop because the pause is from far long enough for any kind of buffer issue.

The issue is not coming from the pause, because your result are clearly wrong too.

>>>>"I double check with your code, and clearly on my side it's wrong"

>>There is no millisecond delay in our code. Have you tried the exact code which we have provided? If yes, you are getting static behavior even if there is no delay. Can we assume like this?

Yes exactly your code! This sentence referred to your code, I don't know if it's static, but for sure it's not dynamic(1) !!

On your screenshots neither !

>>The code which we have attached below is doing some different work for even and odd iteration numbers (which might not take the same time delay to execute). This code exhibits the dynamic behavior at our end. Could you please try to run on your platform and confirm with us (what are you getting? static or dynamic)?

How can you be sure that you odd() and even() are not optimized out ?

Once forever the behavior, you observe in your result is NOT dynamic(1). Even if I have no idea what it's, maybe guide

it's impossible with dynamic(1) that "48 using thread 1" is done and print "before 42 using thread 1"

the thread sleep is required in other cases it cannot be conclusive, computations are too fast

this is my code

code

#include <iostream>
#include <thread>
#include <omp.h>

#define N 20

int main()
{

	int* array=new int[N];
	int* array2=new int[N];
	omp_set_num_threads(4);

	unsigned cpt=0;
	#pragma omp parallel for schedule(dynamic,1) firstprivate(array,array2, cpt)
	for (int i = 0; i < N; i++)
	{
		array[i]=omp_get_thread_num();
		array2[i]=cpt++;
		//printf("Iteration %d handled by thread %d\n", i, omp_get_thread_num());
		std::this_thread::sleep_for(std::chrono::seconds(i%2+1));
	}
	
	for (int i = 0; i < N; ++i)
	{
		printf("Iteration %d handled by thread %d, pos in thread %d \n", i, array[i], array2[i]);
	}
	delete array;
	delete array2;
	return 0;
}

I attach the 3 screenshots. With your two codes and mine of today

[MadhuK_Intel]

Hi,

Thank you for sharing the details, we were able to reproduce your issue. We are working on it. We will get back to you soon.

Thanks and Regards

Madhu.

[DMITRY_T_Intel]

Hi,

I escalated this issue to development. I will update it as soon as I get any information.

Thanks!

[mgravey]

Hi,

I make a discover early this day.

I think that historically "dynamic" implied "ordered" when not anymore.

I didn't have time to have a look to the OMP standard to see what it should be.  But maybe it's right.

Personally, I optimize the code such as it starts by longer job first, and then it doesn't work as you expect if it's unsorted, but maybe it has some application to not enforcing the sorted. Maybe some interesting cash effect. IDK.

Can maybe someone confirm that this is an expect behavior ?

Best,

Mathieu