How to set OMP/MKL thread stack limit on Linux x86_64?

[drMikeT]

I was wondering what is the default value of the stack limit for every OMP/MKL thread when we use MKL functions ? Does it dynamically vary based on some MKL/OMP heuristic or can it be set?

Does it relate to the corresponding stack shell resource limit in effect for the process calling MKL functions ?

At times I would like to let 1 MKL thread be able to use the entire memory of a node but at other times I would like to set OMP_NUM_THREADS equal to the number of cores. In the past, small stack limit would let Fortran code using MKL crash so I increased it to a rather large value. I do not want to have stack limits larger than what they should be though as with very large shell stack limit other multi-threaded processes started having problem lauching threads....

These are the versions of MKL I am interested in (and hopefully soon only for the latter one :) are

MKL v. 10.2.6.038
MKL v. 10.3.10.319

thanks ---michael

[Vladimir_P_1234567890]

Hi Michael,

doesn't usualOMP_STACKSIZE work for you?

--Vladimir

[TimP]

Intel OpenMP default thread stack size is 2MB on 32-bit OS, 4MB on 64-bit OS, including x86_64. As Vladimir mentioned, the portable way to change it is with OMP_STACKSIZE. There is also the kmp_stacksize function call.
Both the shell stack limit and OMP_STACKSIZE need to be sufficient for your job; you must set them separately.

[SergeyKostrov]

OpenMP environment variables could be easily set in a program and here isexample:

[cpp] ... int iRetCode = _putenv( "OMP_STACKSIZE=256K" ); if( iRetCode == 0 ) printf( "OMP_STACKSIZE=%sn", getenv( "OMP_STACKSIZE" ) ); else printf( "Error: Failed to Set Environment Variable OMP_STACKSIZEn" ); ... [/cpp]
>>...Does it relate to the corresponding stack shell resource limit in effect for the process calling MKL functions?

I think No.

>>...small stack limit would let Fortran code using MKL crash...

A minimal size of the stack you can setfor OMP_STACKSIZEis 32K and it is not enoughwhen large data sets are
used for processing.

[TimP]

The only case I have seen for reducing OMP_STACKSIZE from the default size is when a large value of OMP_NUM_THREADS is set, when running on a restricted address space, e.g. 32-bit OS. It's not so common to have an application which uses 16 or more threads effectively which can run in such a restricted space.

[drMikeT]

Quoting Vladimir Polin (Intel)

Hi Michael,

doesn't usualOMP_STACKSIZE work for you?

--Vladimir

Hi Vladimir, thanks for the reminder (even though I don't like to follow the "usual" methods :)

My question was more focused on how the OMP (and MKL) libraries set by default the OMP/MKL stack sizes in the absence of any other information. We experienced mutli-threaded OMP/MKL processes whose resource stack limit was set high, having hard time launching threads.

You are right in the sense one has to set the OMP_STACKSIZE or KMP_STACKSIZE, kmp_set_stacksize_s(), if they know what is a reasonable limit. Alas our users don't bother to set it to anything or they set it soo small or too big.

thanks

Michael

[drMikeT]

Quoting TimP (Intel)

Intel OpenMP default thread stack size is 2MB on 32-bit OS, 4MB on 64-bit OS, including x86_64. As Vladimir mentioned, the portable way to change it is with OMP_STACKSIZE. There is also the kmp_stacksize function call.
Both the shell stack limit and OMP_STACKSIZE need to be sufficient for your job; you must set them separately.

Hi Tim,

thanks for the info. For pure Pthreads the manual says the stack size of a Pthread if not explicitly set is inheritted by the process's resource stack limit and I was wondering if OMP/MKL did the same.

As you mentioned the best way is to set it to whatever is reasonable for you but our users do not set these and I was wondering what is the default OMP/MKL behavior.

-mike

[drMikeT]

Quoting TimP (Intel)

The only case I have seen for reducing OMP_STACKSIZE from the default size is when a large value of OMP_NUM_THREADS is set, when running on a restricted address space, e.g. 32-bit OS. It's not so common to have an application which uses 16 or more threads effectively which can run in such a restricted space.

We were having a problem with the opposite: stack resource limit was set high to accomodate Fortran code with huge stack resident arrays, but the same code then was having problem with launching threads (MPI auxilliary threads).

When the connection of (P)threads used by code not written by us and stack resource limit was made the question was if the OMP/MKL libs would also pick it up and use it for their own threading when OMP_STACKSIZE is not set explicitly.

thanks for the replies --mike

[TimP]

ifort has the -heap-arrays option to change allocations from stack to heap. COMMON blocks currently go on heap (and, in effect, default to SAVE attributes), but local arrays, even in the main program, default to stack, with the options I use. Unlike certain other compilers, if you specify a numeric argument on ifort -heap-arrays, it affects only allocations whose size is known at compile time.

[tomorrowwillbefine]

Microsoft Office 2010 is actually the newest software from microsoft office 2010 keys Microsoft Corporation introduced in the last year. Its leading aims tend to be to catch the present business requirements and to be on top of every competition with regard to the international market criteria. This can be a very good idea to obtain Microsoft Office 2010 Key immediately to maintain norton antivirus keys yourself up-to-date and to present you with the vast qualified progress opportunities for success. Microsoft Office 2010 is available in both 32-bit and 64-bit editions, but attention please the two are not able to co-exist on the very same personal computer. All of the Office 2010 editions are kaspersky antivirus keys suitable for Windows XP SP3, Windows Vista and Windows 7.

www.keyyeah.com