Paging file too small

[Bruce_Weaver]

I don't want to page!

I've got lots of Ram (128 GB) while running latest Fortran in 64 bits but I get

OMP: Cannot create Thread

OMP: System Error #1455 Paging file is too small for this operation to complete

OMP: Error 178: function GetExitCodeThread failed

OMP: System error #6  The handle is invalid

The problem occurs when the program reaches the multi-threaded section.

I've got lots of ram because I don't want to page this program that runs for days.  I have set the virtual memory to zero to force the executing code to use the memory instead of paging to disk.  Reducing the number of threads seems to help but I need to use all the threads I can get.  If all the threads were to run, there would still be lots of memory left.  Making a non-zero paging file area also allows all the threads to run.  Who is to blame & How do I fix it?  I.e., how do I force the program to use ram instead of paging?

thanks for  any suggestions

[Xj_Kong]

Have you tried to restart? 128 GB RAM is really big enough but depends on what operation in your program is running or what data file is loading...

Good luck!

[andrew_4619]

More threads doesn't mean faster! You would expect some point where more threads = slower. there will be an optimum. I guess just because you have a page size it doesn't mean the application will need to use it. I personally would allow paging and make some tests to see if paging occurs and what the optimum parameters are for speed....

[Bruce_Weaver]

Hi Andrew,

That's a different issue.  My code is designed to use threading very efficiently, is back-end bound (very ALU-bound) and maxes out at about 3/4 the number of possible threads -- about what you'd expect with two threads sharing each ALU.  With all the memory, why should ANY paging be demanded?

[jimdempseyatthecove]

Did you do anything silly regarding the stack settings (Linker)? Declare you want to give all threads everything or something really huge?

Second suggestion:

Using 0 for Virtual Memory may not provide for your application to use all of RAM. The default may be (???) some smaller portion of this. Searching msdn might provide information as to the default behavior (per process) with a 0 setting. I suggest setting the Page File size to something like 2x Physical RAM. If your program is well mannered (not doing something silly), then after a short startup period, the program should not exhibit paging. At least you can try this and observe: a) if you ruin out of page file (silly), or b) what the footprint is if the program runs (IOW .gt. physical RAM-O/S and .lt. page file size - O/S).

Jim Dempsey

[Bruce_Weaver]

will do search and try.  I guess you're suggesting that WIN7 needs the paging to set up the code (or OMP, which seems to be   the source of the complaint) to run but then doesn't use it during execution.  No I don't give all the data to each thread,otherwise I'd use co-arrays instead OpenMP.  Actually each thread has very  little private data.  I suspect that Intel's implementation of Open MP assumes paging and uses it even if it is not needed.

[jimdempseyatthecove]

An application (sans making API calls to the O/S) is oblivious to paging. The Virtual Machine of the process thinks it has all the addressing capacity of the CPU. The CRTL heap manager, interacts with the O/S to map and specify which addresses will be valid (though not necessarily immediately assigned to the page file). This can result in an allocation succeeding, but a application failing sometime later when first touches the unallocated page address (presumably by write). This causes a page fault to the O/S, and then the O/S allocates the page from the page file and assigns it to the page of the process (it may also wipe the page too, to keep an application from snooping around).

OpenMP uses the CRTL heap manager (unless you replace it with something else).

What happens to the memory footprint when you insert into the start of the program (as the first parallel region):

PAUSE "Check memory footprint before"
!$OMP PARALLEL
print *,"hello from thread", omp_get_thread_num()
!$OMP END PARALLEL
PAUSE "Check memory footprint after"

If that crashes, then set your environment variable to restrict OpenMP to use but 2 threads.

Then re-run and note the additional memory for the one additional thread.

Jim Dempsey

[andrew_4619]

I would very much doubt OMP requested "paging" that is the job of the OS. You ask for memory and the OS supplies in in the way It see fit.