Intel® oneAPI Threading Building Blocks
Ask questions and share information about adding parallelism to your applications when using this threading library.

malloc proxy and static variables

soth
Beginner
1,017 Views
Hi,
I`ve run into a problem with the malloc proxy and static variables (MSVS 2008, Win7). The problem is that tbb replaces memory allocation routines when the DllMain function is run, but this happens after the static variable initialization phase and they use the default allocation functions, but when the program finishes those static variables are destroyed with substituted routines, which sometimes leads (i wonder why not always?) to a crash. The program is most likely to crash when the static variable aquired some kind of resource (file or something else) during initialization and kept it until the program has finished. I think that if we want malloc proxy to work correctly, we must revert the memory function substitution done in DllMain when the dll is being detached from the process(it happens before static variables are destroyed), so that static variables could deallocate the memory using the same functions with which that memory was acuired.
0 Kudos
1 Solution
Alexey-Kukanov
Employee
1,017 Views
Quoting soth
So I have a feature request to get a function in tbb_malloc library, which tells if the memory was allocated with tbb allocator (basically such method already exists, and it is the isRecognized function from MemoryAllocator.cpp, i`ve added it to the library`s exported functions to make thing work for me).

While isRecognized works fine as part of our malloc replacement mechanism, making it generally available with the possibility to pass an arbitrary address to it is dangerous. Meanwhile we were unable to find similar functionality in other allocators, such as Hoard and the "standard" C runtime allocators in MSVC and glibc. So we are unlikely to make this feature. I hope you are fine with modifying the GPLed version of TBB for your purposes.

View solution in original post

0 Kudos
10 Replies
Vladimir_P_1234567890
1,017 Views
hi soth,

which tbb version do you use? 2.2 update 1 andnewer should have better support for the windows malloc replacement including static objects support improvements.

--Vladimir
0 Kudos
soth
Beginner
1,017 Views
hi soth,

which tbb version do you use? 2.2 update 1 andnewer should have better support for the windows malloc replacement including static objects support improvements.

--Vladimir
It`s tbb 2.2, i dont know for sure(not at home right now), but most likely it is the tbb22_20090809oss version. I need to update to get correct behavior?

I`ve looked into the proxy.cpp file from the develepment release 20091101 and it seems to save the addresses of original free() functions, but i dont get when and where these functions are replaced back. Could someone explain it please?
0 Kudos
soth
Beginner
1,017 Views
Quoting - soth
It`s tbb 2.2, i dont know for sure(not at home right now), but most likely it is the tbb22_20090809oss version. I need to update to get correct behavior?

I`ve looked into the proxy.cpp file from the develepment release 20091101 and it seems to save the addresses of original free() functions, but i dont get when and where these functions are replaced back. Could someone explain it please?
Ah, nevermind, I`ve figured out how this works.
0 Kudos
Vladimir_P_1234567890
1,017 Views
Quoting - soth
It`s tbb 2.2, i dont know for sure(not at home right now), but most likely it is the tbb22_20090809oss version. I need to update to get correct behavior?

I`ve looked into the proxy.cpp file from the develepment release 20091101 and it seems to save the addresses of original free() functions, but i dont get when and where these functions are replaced back. Could someone explain it please?

Is this the 64-bit system? There is known issue with crash on application exit on 64-bit vista or win7 for this development release. If yes pls take the newer package.

Original functions are not replaced back. Even after final call of DLLmainproxy libaryis not unloaded and replaced functions can be called and executed.

--Vladimir
0 Kudos
soth
Beginner
1,017 Views

Is this the 64-bit system? There is known issue with crash on application exit on 64-bit vista or win7 for this development release. If yes pls take the newer package.

Original functions are not replaced back. Even after final call of DLLmainproxy libaryis not unloaded and replaced functions can be called and executed.

--Vladimir
I have a 32 bit system. Can ReplaceFunction be used to replace substituted functions back, given i have the address of the original function? I`m asking that because in my project i use tbb_function_replacement.h/cpp and modified proxy to inject my own allocation routines, which in turn may use tbb or other allocators, so i dont have a magic number for my pointer to detect whether it was allocated by me or by msvs crt, i think that switching back to original crt functions when the dll detaches would solve the problem with static objects.
0 Kudos
Vladimir_P_1234567890
1,017 Views
Quoting - soth
Can ReplaceFunction be used to replace substituted functions back, given i have the address of the original function?
It is tricky but I do not seea reason why it shouldn't work in case you do not unload proxy manually.

BTW in your case you cannot use realloc to resize data structures in static objects in the middle of the program.

--Vladimir
0 Kudos
soth
Beginner
1,017 Views
Hi, I`ve been playing around with tbb function replacement facilities again to patch the msvc crt libraries from my own dll and realized that just switching back to original allocation routines when the dll is detached from the process does not work for static objects, because they internally could allocate memory using patched functions during the program lifetime and free them when the dll is already detached, so it seems the correct way to do patching (as tbb_malloc_proxy and google tcmalloc does) is saving the original deallocation functions and using them when it is detected that the pointer was not allocated by the patched allocator. So I have a feature request to get a function in tbb_malloc library, which tells if the memory was allocated with tbb allocator (basically such method already exists, and it is the isRecognized function from MemoryAllocator.cpp, i`ve added it to the library`s exported functions to make thing work for me).

0 Kudos
Alexey-Kukanov
Employee
1,018 Views
Quoting soth
So I have a feature request to get a function in tbb_malloc library, which tells if the memory was allocated with tbb allocator (basically such method already exists, and it is the isRecognized function from MemoryAllocator.cpp, i`ve added it to the library`s exported functions to make thing work for me).

While isRecognized works fine as part of our malloc replacement mechanism, making it generally available with the possibility to pass an arbitrary address to it is dangerous. Meanwhile we were unable to find similar functionality in other allocators, such as Hoard and the "standard" C runtime allocators in MSVC and glibc. So we are unlikely to make this feature. I hope you are fine with modifying the GPLed version of TBB for your purposes.
0 Kudos
soth
Beginner
1,017 Views
Yes, I`m fine with modifying the sources.
0 Kudos
Long_Nguyen
Beginner
1,017 Views

Hi Soth,

Could you please tell me what you did to get rid of the crash? I run into the same situation (http://software.intel.com/en-us/forums/showthread.php?t=72457) and my app still crashes even after I tried to override the global new and delete operator with calls to scalable_malloc and scalable_free, totally bypassed tbbmalloc_proxy.dll and the function replacement stuff. I even exposed the isRecognized function as you suggested.

if (isRecognized(addr))
{
scalable_free(addr);
}
else
{
::operatordelete(addr); // call the default delete operator
}

After I made this change (link with tbbmalloc.lib directly, bypassing tbbmalloc_proxy.dll and memory function replacement), my app crashes HARD with access violation in the C runtime FreeHeap function.

Thanks for your help.

-Long

0 Kudos
Reply