I encountered the following error when using "parallel studio xe 2019 update3 cluster edition" on the cluster: forrtl: severe (41): insufficient virtual memory. The same computing nodes and jobs can run normally using "parallel studio xe 2017 update5 cluster edition". How can I solve this problem?
The attachment is a detailed error message
Thanks for reaching out to us.
Please try monitoring your program with both of the PSXE versions and observe the memory usage, let us know if there large memory difference between them.
It seems that somehow your program is trying to use more than available memory.
Also, send us a small reproducer for this issue so that we can test it out on our side.
Sorry for the delay, please send us a small reproducer to get more insight into your issue.
You can simply run the top command on each node to check your RAM utilization while executing your code.
So sorry for the delay.
It will be difficult for us to review this much large project. If you have a small reproducer please feel free to send it with this thread itself.
We think we got the real cause of your issue. Please check the output of top command, and make a note of the virtual memory usage by each of your processes.
The MPI library is using a shared virtual memory region within a node, which is getting mapped into each process’s virtual address spaces, so when the virtual memory consumption is getting queried for any of the ranks, the size of the entire memory region shown. That might be the main reason for insufficient virtual memory in your case with PSXE 2019, and not in PSXE 2017.
Please follow the below article link for more details.
And try comparing the output from top and actual memory utilized, so if this is the case with your application, then the cause of the issue is more likely not from Intel MPI Library.
Hope the provided details will help you to solve your issue. Do let us know your findings and if you have any issues do let us know.
Thanks for the reply.
Will you please post a new thread in this forum after conducting your test?
You can mention the link to this thread whenever you will post back to us with your findings. This will help us to monitor your issue more conveniently.
Thanks a lot for the prompt reply.
Please do post a new thread whenever you will conduct your tests. We will try to resolve all of them.
As of now we will stop monitoring this thread and will wait for your findings.