Win32 API function 'CreateThread' fails with system error code 8 - Resolved

[SergeyKostrov]

I detected an issue for a Win32 application that uses TBB version 4.

Win32 API function 'CreateThread' fails with system error code 8 when a 'tbb_thread' object creates
a thread and 2GB of memory is reserved\\committed for the Win32 application.

Here are some technical details:

Error Code: 8
Decription: Not enough storage is available to process this command
Define : ERROR_NOT_ENOUGH_MEMORY

IDE: Visual Studio 2005

I setHeap( Reserve\\Commit ) and Stack ( Reserve\\Commit )values to 536,870,912 bytes in [ Project Properties -> Linker -> System ]:

Heap Reserve Size: 536870912
Heap Commit Size: 536870912
Stack Reserve Size : 536870912
Stack Commit Size: 536870912

In total 2GB of memory is reserved\\committed for the Win32 application.I need this for a multiplication oftwo large matrices.

Note 1: This is NOT a problem with TBB. This is some issue, possibly known, with the Win32 API function 'CreateThread'.

Note 2: An MSDN's article about 'Address Windowing Extensions' states:

...
...The address space is usually split so that 2 GB of address space is directly accessible to the application...
...

[RafSchietekat]

Note that those values don't add up to 2 GB (and I'm not a Windows programmer): initially you would have the heap reserve plus the stack reserve (here 1 GB), and then it depends on the operating system how the commit values are used.

But I don't see any need for such a large stack size, and the problem might be that it may be applied to each additional thread for an impossible total memory consumption (unless perhaps if you override it when creating those threads).

[SergeyKostrov]

>>...But I don't see any need for such a large stack size...

It is off the subject, but here are some details...

In case of multiplication of two matrices with sizes greater than 2048 x 2048:

Strassen's Stack-based algorithm uses "enormous" amount of memory from the Stack

Strassen's Heap-based algorithm uses "enormous" amount of memory from the Heap

[jimdempseyatthecove]

If you are running a 32-bit application on Windows (without LARGE_ADDRESS)
Then the low half of virtual address space (32-bits) is available for application. (2GB)
And the high half of the virtual address space is available for the system (Windows).
With LARGE_ADDRESS, the low 3/4ths of the is available for applicaiton (3GB),
and the high 1/4th is available for the system (Windows).

The application space is:

code
static data
heap
stack
(nAdditionalThreads * (context + stack))

Thread-0: code + static data + heap (0.5GB shared) + stack (0.5GB thread private)
Thread-1: +context (few KB) + stack (0.5GB thread private)
Thread-2: +context (few KB) + stack (0.5GB thread private)
...

At what point are you going to exceed 2GB? (or 3GB?)


Jim Dempsey

[SergeyKostrov]

>>...At what point are you going to exceed 2GB? (or 3GB?)

It will exceed 2GB:
In case of a matrix multiplication using Strassen's algorithm ( Stack- or Heap-based ) if matrices are
greater than 2,048 x 2,048 with a threshold value 32 x 32 ( for data types 'float''double' ).

It will exceed 3GB:
In case of a matrix multiplication using Strassen's algorithm ( Stack- or Heap-based ) if matrices are
greater than 4,096 x 4,096 with a threshold value 32 x 32 ( for data types 'float''double'\
'long double' ).

>>...With LARGE_ADDRESS, the low 3/4ths of the is available for applicaiton (3GB),
>>and the high 1/4th is available for the system (Windows)...

I know this.32-bit Windowshas to bestarted with the /3GB option. Here is a link to Microsoft's Technet
article:

http://technet.microsoft.com/en-us/library/bb124810(EXCHG.65).aspx

...

The /3GB switch is supported only on the following operating systems:

Windows 2000 Advanced Server
Windows 2000 Datacenter Server
Windows Server 2003 Standard Edition
Windows Server 2003 Enterprise Edition
Windows Server 2003 Datacenter Edition
...

Unfortunately, Windows XP is not on the list.

[SergeyKostrov]

This isa follow up.

After some testing I decided to change Heap ( Reserve\Commit ) and Stack ( Reserve\Commit ) values to:

Heap Reserve Size: 1069547520
Heap Commit Size: 1069547520
Stack Reserve Size: 4194304
Stack Commit Size: 4194304

1069547520 = ( 2^31 - ( 2^22 * 2 ) ) / 2
4194304 = 2^22

Here is some information on aStress-Testing of TBB with theseHeap ( Reserve\Commit ) and Stack ( Reserve\Commit ) values:

Debug Configuration : up to 233 threads could be created
ReleaseConfiguration: up to 242 threads could be created

Best regards,
Sergey