OpenMP won't run without shared memory (/dev/shm)

michaelkonecny · ‎09-21-2023

Some time between 2020 and 2022, OpenMP in Intel MKL has started using /dev/shm on Linux.

This makes it impossible to use MKL in environments that do not have SHM, such as AWS Lambda: https://aws.amazon.com/blogs/compute/parallel-processing-in-python-with-aws-lambda/

OMP seems to have started using SHM in October 2020, when this commit to LLVM made the change to use it instead of environment variables to store its initialization flags.

It also seems to have already been fixed in LLVM in January 2023, see this review.

However, the latest version of Intel MKL (2023.2.0) still requires SHM in order to run (crashes with a fatal error otherwise).

I'm guessing a newer version of OpenMP will make it from LLVM into intel eventually.

My question to Intel is: can you give an estimate on when this could be?
Could you perhaps describe the cycle in which new versions of LLVM are propagated into MKL?

Thank you!

Ron_Green · ‎09-21-2023

Moving this thread to the MKL forum

AryanK_Intel · ‎09-25-2023

Hi Michaelkonecny,

Thanks for posting in Intel Communities.

Could you please share your OS, hardware details along with complete sample reproducer code along with the steps you are following so that we can reproduce the issue from our end and investigate more?

Best regards,

Sri Raj Aryan.K

michaelkonecny · ‎09-25-2023

Hello,

I don't need you to reproduce the issue.

I need someone who understands how changes in OpenMP get propagated into MKL to read my question and answer it.

AryanK_Intel · ‎09-25-2023

Hi Michaelkonecny,

We are working on your issue. We will get back to you soon.

Best regards,

Sri Raj Aryan.K

AryanK_Intel · ‎10-03-2023

Hi Michaelkonecny,

>>Can you give an estimate on when this could be? Could you perhaps describe the cycle in which new versions of LLVM are propagated into MKL?

Thanks for sharing your feedback. However, unfortunately we can't consider your suggestions for MKL as of now as Intel MKL doesn't support it.

Thanks and Regards,

Sri Raj Aryan.K

michaelkonecny · ‎10-03-2023

Hello,

there was no "suggestion" in what I wrote, I was merely describing a problem and asking a question about it.

I'm not sure what you mean by "Intel MKL doesn't support it", that doesn't make any sense.

I'm sorry to say this, but so far, your responses have been of absolutely no use and seem to lack the most basic understanding of what I'm asking.

Can you get me in touch with somebody who at least understands my questions?

Thank you,

Michael Konecny

Maria_K_Intel · ‎10-04-2023

Hello,

The fix for Intel OpenMP will be available in 2024.0 release.

michaelkonecny · ‎10-06-2023

Terrific, thank you!

VarshaS_Intel · ‎10-16-2023

Hi,

It’s great to know that the issue has been resolved, in case you run into any other issues please feel free to create a new thread.

Have a Good Day!

Thanks & Regards,

Varsha