The title of the thread describes the problem with MKL and I'll provide additional technical details.
From the attached picture, it looks the memory (virtual memory) used by the applications is close to limitation. The max size about 130G for system and application almost use that size.
Also from the performance point of view, the virtual memory is far beyond the physical memory (32G) now. It may have much memory swapping, can create some bad performance.
Hi Chao, I've provided example and lots of technical details in order to avoid any talks which I would rate as "speculative". Please try to reproduce the problem. On my side, I'll try to increase VM Max size and don't worry about performance of calculations since this is another issue Not related to the problem. Thanks in advance.
I have a quick run here, see one similar error report. MATMUL is the Fortran intrinsic function, not a call to the MKL function, so the problem is not likely to be related to the scalable_malloc/mkl_malloc functions, but be related with the MATMUL function, and this function is auto-paralleled when compiling with /Qparalle witch.
I will move this thread to the Fortran forum, so more expert there could help. A quick summary for the problem:
When compiling with /Qparallel for the fortran intrinsic function MATMUL, it will report the bellow errors, even with the small matrix:
OMP: System error #1455: The paging file is too small for this operation to complete
The test code in the first comment could show the problem.
Look at your screen shot 3. You circled the commit size of your Fortran app and circled the page file size to show the app used less page file than the total size. You forgot to include the commit sizes for devenv, WLTray, explorer, ... This gets you to ~130.583GB, then you must add the pageble portions of the Services (not shown on the tab). Also not shown in your commit number is the size of the failing allocation.
At least the system is taking a graceful exit in reporting to the application that it does not have the resources to create the thread. Some OS's will choke down and crawl at this point. (no error back to the app, no progress for anything else on the system)
Your experiment is pushing the system to break your app. Well, it did. Go out and buy a 256GB SSD and reserve all of that for your page file.
If you look at your 8 (9) thread screen shot at 27,320,672
And your 16 (17) before crash at 44,125,712
Subtract the 9 from 17 gives 16,805,040 for 8 additional threads
Divide by 8 gives 2,100,630 required per thread.
Your error message said omp could not add a thread when at 130,282,476 (+2,100,630 = 132,383,106) this exceeds your page file size.
Now, if we take 9 x 2,100,630 = 18,905,670 off the total for 9 threads we get 8,415,002KB for heap and program. This is in line with your 9GB estimate.
What is this extra thread, is it one you are spawning or a watchdog of OpenMP?