Intel® MPI Library
Get help with building, analyzing, optimizing, and scaling high-performance computing (HPC) applications.
2224 Discussions

About parallel studio xe 2019 update3 cluster edition

周__浩
Beginner
2,749 Views

Hi administrator

 

I encountered the following error when using "parallel studio xe 2019 update3 cluster edition" on the cluster: forrtl: severe (41): insufficient virtual memory. The same computing nodes and jobs can run normally using "parallel studio xe 2017 update5 cluster edition". How can I solve this problem?

The attachment is a detailed error message

 

 

0 Kudos
11 Replies
AbhishekD_Intel
Moderator
2,725 Views

Hi,


Thanks for reaching out to us.

Please try monitoring your program with both of the PSXE versions and observe the memory usage, let us know if there large memory difference between them.

It seems that somehow your program is trying to use more than available memory.


Also, send us a small reproducer for this issue so that we can test it out on our side.



Warm Regards,

Abhishek



0 Kudos
周__浩
Beginner
2,717 Views

Thank you very much for your reply. How do I call the PSXE monitoring program?

0 Kudos
周__浩
Beginner
2,715 Views

Hi

Parallel studio xe 2019 update3 cluster edition has failed, and I cannot check the memory usage. Parallel studio xe 2017 update5 cluster edition is still running normally, the memory usage is in the attachment.

0 Kudos
AbhishekD_Intel
Moderator
2,668 Views

Hi,


Sorry for the delay, please send us a small reproducer to get more insight into your issue.

You can simply run the top command on each node to check your RAM utilization while executing your code.


Warm Regards,

Abhishek


0 Kudos
周__浩
Beginner
2,663 Views

Hi

Can I send an email? The file is 183MB and I cannot upload it to the forum.

0 Kudos
AbhishekD_Intel
Moderator
2,610 Views

Hi,


So sorry for the delay.

It will be difficult for us to review this much large project. If you have a small reproducer please feel free to send it with this thread itself.

We think we got the real cause of your issue. Please check the output of top command, and make a note of the virtual memory usage by each of your processes.

The MPI library is using a shared virtual memory region within a node, which is getting mapped into each process’s virtual address spaces, so when the virtual memory consumption is getting queried for any of the ranks, the size of the entire memory region shown. That might be the main reason for insufficient virtual memory in your case with PSXE 2019, and not in PSXE 2017.


Please follow the below article link for more details.

https://software.intel.com/content/www/us/en/develop/articles/evaluating-virtual-memory-consumption-in-intel-mpi-library.html

And try comparing the output from top and actual memory utilized, so if this is the case with your application, then the cause of the issue is more likely not from Intel MPI Library.


Hope the provided details will help you to solve your issue. Do let us know your findings and if you have any issues do let us know.



Warm Regards,

Abhishek


0 Kudos
AbhishekD_Intel
Moderator
2,599 Views

Hi,

Please give us an update on the provided details.


Warm Regards,

Abhishek


0 Kudos
周__浩
Beginner
2,588 Views

Hi.

I'm very sorry, my business trip is not over yet. The test has not yet been completed. If there is progress, I will contact you.

0 Kudos
AbhishekD_Intel
Moderator
2,576 Views

Hi,


Thanks for the reply.

Will you please post a new thread in this forum after conducting your test?

You can mention the link to this thread whenever you will post back to us with your findings. This will help us to monitor your issue more conveniently.


Warm Regards,

Abhishek


0 Kudos
周__浩
Beginner
2,572 Views
0 Kudos
AbhishekD_Intel
Moderator
2,571 Views

Hi,


Thanks a lot for the prompt reply.

Please do post a new thread whenever you will conduct your tests. We will try to resolve all of them.

As of now we will stop monitoring this thread and will wait for your findings.


Thank You,

Abhishek


0 Kudos
Reply