Intel® Fortran Compiler
Build applications that can scale for the future with optimized code designed for Intel® Xeon® and compatible processors.

OMP Parallel memory allocation

tihomir
Beginner
774 Views

Hi

My program sometimes runs out of memory and can not start additional threads. The program terminates at the

$OMP Parallel

line with the error message

system error(8): __kmp_create_worker: CreateThread: Not enough storage is availa
ble to process this command.

OMP abort: fatal system error detected.

Is there a way to capture this exception and continue the computation with a single thread since no additional threads can be started rather then quit altogether? Thank you

Tihomir

0 Kudos
8 Replies
jimdempseyatthecove
Honored Contributor III
774 Views

Check your application to see if it is recursively creating threads. For this error to occure you application would be creating 100's or 1000's of threads (provided this is an operating system error). OpenMP also has a maximum number of threads (setable by environment variable or openmp function call or defaults to number of processors). If you have nested parallel regions (possibly by recursive programming) then you will consume the available number of theads.

Assuming no programming error, you can specify a higher maximum number of threads and specify dynamic threading is to be used.

Jim Dempsey

0 Kudos
tihomir
Beginner
774 Views

Jim

Thanks for the reply. This happens with 2 threads, on a two core machine. Thereare no nested threads or recursive algorithms. When I use smaller data set (using less memory) the algorithm runs fine.

Actually I need to capture the exception not only to improve the algorithm but also toprovide my users with informative message ... right now the program just collapses. Virtually every system command in Fortran has an error indicator ... but I can find this possibility for the OMP PARALLEL command.

Thanks

Tihomir

0 Kudos
jimdempseyatthecove
Honored Contributor III
774 Views

If on first call, it sounds like one of two problems:

a) You are linking in the wrong run time library
b) By the time the 1st parallel region is reached, all the virtual memory has been allocated.

As a test for both, and possible correction for b)

In the start of your application (before you allocate much memory)insert a a little test section that simply starts a parallel secton, prints the thread number, exits the parallel section, resumes with your application...

If you get the error message in the new sectionthen this indicates runtime library mixup.

If the error goes away, then I suspect your application is gobbling up all of virtual memory (preallocating objects to maximum it can get).

If the error appears on the original $OMP section then there is something else happening. The tread is alredy running (assuming you $OMP section is not recursive). Once an extrathread is created for use by OpenMP the thread persists for the duration of the application and gets reused.

Jim

0 Kudos
tihomir
Beginner
774 Views

Jim

The problem is b). The memory the programis supposed to use is a bit short of2 GB (which is what I have). So as the system is preparing to set a new thread going it runs out of memory. I am not sure how much memory is needed to start a new thread but if it is not trivial (as it seems) perhaps there should be an indicator to return a memory allocationerror rather than crash. Thanks.

Tihomir

BTW, who is in charge of the OMP implementation? Is it Intel or openmp.org?

0 Kudos
Steven_L_Intel1
Employee
774 Views
The implementation is Intel's.
0 Kudos
jimdempseyatthecove
Honored Contributor III
774 Views

Tihomir,

Did starting the thread(s) prior to application allocation fix the thread startup problem?

Note, each additional thread started, unless indicated otherwise (KMP_STACKSIZE),assumes a stack size equal to the application main (linker option).

Too often a programmer is tempted to specify a humoungous stack size to avoid stack overflow problems. Then later forget about this when they parallel-ize. Memory management is not within the scope of OMP implimentation. You could write your own code to check on available contiguousmemory, and if sufficient run the parallel code, if not run the serial code. Excessive stack consumption can usualy be cured with some code change (allocatable or pooled memory blocks).

This could be automated by the implementation. However... you went through the bother to add $OMP parallization code to your application. Isn't it worth the little extra effort to get the parallel code functioning rather than never working?

Jim Dempsey

0 Kudos
tihomir
Beginner
774 Views

Great!!! This did the trick. Checking that there is enoughmemory for the stack catches the exception. BTW, do we have a variable inside FORTRAN that matches the stacksizefrom the Linker options for the main thread (unrelated to OMP)? Thank you!!!

0 Kudos
jimdempseyatthecove
Honored Contributor III
774 Views

If the application is OpenMP ...

Stack Size

In most cases, environment variables can be used in place of
the extension library routines. For example, the stack size
of the parallel threads may be set using the
KMP_STACKSIZE
environment variable rather than the KMP_SET_STACKSIZE() or
KMP_SET_STACKSIZE_S() library routine.

Note

A run-time call to an Intel extension routine takes precedence
over the corresponding environment variable setting.
The routines KMP_SET_STACKSIZE() and KMP_GET_STACKSIZE() take
a 32-bit argument only. The routines KMP_SET_STACKSIZE_S() and
KMP_GET_STACKSIZE_S() take a SIZE_T argument, which can hold
64-bit integers.
On Itanium-based systems, it is recommended to always use
KMP_SET_STACKSIZE_S() and KMP_GET_STACKSIZE_S().
These _S() variants must be used if you need to set a
stack
size = 2**32 bytes (4 gigabytes).
You can link in OMP_LIB in single thredded applications.

For non-OpenMP use there is a C RTL function __chkstk. However, it is recommended that you not use this as it will force the allocation of the virtual memory. No need to gobble it up if the application doesn't need it.

Seach ms for other techniques.

Jim Dempsey

0 Kudos
Reply