- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
We have encountered several major memory leaks in Siesta, CP2K and QE when using D&C eigensolver.
See e.g. this thread: https://gitlab.com/siesta-project/siesta/-/issues/29#note_735026816
with explicit showcasing that MPI causes some problems.
Whether this is related to the MKL implementation, or the MPI implementation is not clear to me.
Elsewhere I found these topics:
https://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/Memory-leak-in-dpotrf-and-dpotri/m-p/1188766
https://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/Memory-Leak-in-MKL/m-p/1153543
Where it is suggested to insert a call:
mkl_free_buffers
I have to say that this is not really a good solution? Why can't MKL garbage collect buffers not reachable?
/ Nick
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Thanks for reaching out to us.
Could please let us know with which oneMKL version you have tried along with your OS details?
For the issue which you have mentioned regarding memory leaks in dpotrf & dpotri, this issue has been fixed in 2021.3.0 release.
Here is the article where you can get information regarding the bug fixes
>> memory leaks in D&C eigensolver parallel.
You can try the latest oneMKL version which is 2021.4.0 and see if it resolves the issue regarding memory leaks.
If the issue still persists even with the latest release, please provide us with a minimal reproducer(& steps if any) so that we can work on it from our end.
Please confirm the observed memory leaks by checking them using Intel Inspector as well.
It would be helpful if you can summarize once again here with the versions of MKL & MPI where no memory leaks were observed.
Regards,
Vidya.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
OS details: Red Hat Enterprise Linux Server release 7.6 (Maipo)
Summary of the versions of MKL & MPI for Siesta with D&C eigensolver:
intel/2018u4 + intel-mpi/2018.4.274 + mkl/2018u4 | No memory leaks |
intel/2019u4 + intel-mpi/2019.5.281 + mkl/2019u4 | Memory leaks |
oneapi/compiler/2021.3 + oneapi/mpi/2021.3.0 + oneapi/mkl/2021.3.0 | Memory leaks |
intel/2018u4 + intel-mpi/2018.4.274 + openblas-0.3.17 + scalapack-2.1.0 | No memory leaks |
intel/2019u4 + intel-mpi/2019.5.281 + openblas-0.3.17 + scalapack-2.1.0 | Memory leaks |
oneapi/compiler/2021.3 + oneapi/mpi/2021.3.0 + openblas-0.3.17 + scalapack-2.1.0 | Memory leaks |
It looks like the problem is not in MKL, since the same behavior is observed with OpenBLAS + ScaLAPACK as with MKL.
I will try the latest release (oneAPI 2021.4.0) and if the issue persists I will provide the input files to reproduce it.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Reminder:
Could you please provide us with the above-mentioned details (in my previous post) so that we can work on it from our end?
Regards,
Vidya.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Vidya,
Here are the answers to your previous post:
> Could please let us know with which oneMKL version you have tried along with your OS details?
oneMKL version: 2021.3.0
OS details: Red Hat Enterprise Linux Server release 7.6 (Maipo)
> You can try the latest oneMKL version which is 2021.4.0 and see if it resolves the issue regarding memory leaks.
I'm working on installing 2021.4.0 and recompiling and testing Siesta.
> If the issue still persists even with the latest release, please provide us with a minimal reproducer(& steps if any) so that we can work on it from our end.
The minimal reproducer is attached. It contains Siesta makefile (arch.make), job submission script (submit.sh), and Siesta input files (WATER.fdf, O.psf, H.psf). Siesta source code can be downloaded here:
https://gitlab.com/siesta-project/siesta/-/releases/v4.1.5/downloads/siesta-4.1.5.tar.gz
> Please confirm the observed memory leaks by checking them using Intel Inspector as well.
So far I've been using /proc/PID/smaps files in Linux to track memory usage of Siesta. I will confirm with Intel Inspector as well.
> It would be helpful if you can summarize once again here with the versions of MKL & MPI where no memory leaks were observed.
There were no memory leaks with Intel Parallel Studio XE Cluster Edition 2018.4. There were memory leaks with Intel Parallel Studio XE Cluster Edition 2019.5 and with Intel oneAPI HPC Toolkit 2021.3.0.
Regards,
-Georgy
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Thanks for providing the details.
We have reported this issue to the development team, they are looking into this issue. We will get back to you soon.
Regards,
Vidya.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
>>I'm working on installing 2021.4.0 and recompiling and testing Siesta
The issue is fixed in oneAPI 2021.4.0
Could you please try with 2021.4.0 Intel MPI and let us know if your issue is resolved?
Regards,
Vidya.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Vidya, thank you for the update. I just finished testing 2021.4 and I can confirm that the memory leaks are gone. There is one more issue though. CP2K built with 2021.4 freezes after running for several hours, while CP2K built with 2018.4 works fine and reaches the walltime limit. This is not a memory leak as the memory consumption is low. Not sure if this is specific to my OS. I will run some more tests with 2021.4 and if the issue persists I will provide a minimal reproducer (or open a new ticket).
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
>> .....(or open a new ticket)
Yes, you can raise a new thread if you face any issues with respect to CP2K application (as it would be easier to keep track of the issue).
>>I can confirm that the memory leaks are gone
Thanks for the confirmation.
As your issue regarding the memory leaks in the siesta application is resolved, we are closing this thread. Please post a new question if you need any additional information from Intel as this thread will no longer be monitored.
Regards,
Vidya.
![](/skins/images/7FC17B7B85029576C25F1E43CE255B51/responsive_peak/images/icon_anonymous_message.png)
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page