Redefining memory functions in libirc?

[djunglas]

Hello,

I am linking my application with various Intel libraries. In my application I need to control any call to malloc() and friends. That is, I don't want to use the system's malloc() but want to use my own memory allocator. For the MKL libraries I can redirect the memory allocation using i_malloc and friends as described here https://software.intel.com/sites/products/documentation/doclib/iss/2013/mkl/mkl_userguide_lnx/GUID-751CE7F1-BA6A-475F-8B59-CB801F9F1AC3.htm.

I checked the other libraries I am linking with and none of them seems to have reference to memory allocation functions, with the single exception of libirc.a. This library apparently references malloc() etc. to define functions like _mm_malloc(). Is there a way to redefine the memory functions there in a similar way to the MKL libraries? I am not using any of the _mm_malloc() functions in my code (I need libirc.a for other purposes) so my code would not generate references to them. But I am not sure whether other functions in Intel libraries would call those functions?

In summary, I need to make sure one way or the other that linking with libirc.a does not produce direct calls to malloc() at runtime.

A few more notes:

• It is not an option to use LD_PRELOAD or similar techniques to replace references to malloc() at runtime
• All libraries must be linked statically
• I could do something like 'objcopy --redefine-sym malloc=my_malloc libirc.a mylibirc.a' and then link with mylibirc.a but I am not sure whether this would violate any licenses? Also, this is rather ugly and I would like to avoid doing that. Similarly, using ld's --wrap=malloc feature could come to my rescue, but again, this is not something I like to do.

Thank you for any help or hints,

Daniel

[KitturGanesh]

Hi Daniel,
Not aware of any formal mechanism for what you want. You could try creating your own version of malloc(size_t s) and link that first and probably put an assert in your implementation when it's called may be for any intrinsic you use from libirc?  
_Kittur

[djunglas]

Thanks for your tip but I am afraid that this is not going to work in my situation: some of the functions to which I want to redirect allocation may end up calling malloc() themselves (and this is fine in this specific place). So defining my own malloc() would result in an infinite recursion. What I did for now and which has worked so far was to explicitly ignore the object file defining _mm_malloc() during linking.

[KitturGanesh · Accepted solution]

@djunglas,  thanks for the feedback and letting us know on the workaround you used. I did pass that on to the developers with whom I did discuss your situation and they pretty much gave the same feedback I mentioned to you earlier, as there's no formal mechanism to do so.

_Kittur

[djunglas]

Thank you for officially confirming that there is no formal way to redirect memory allocations.

I am fine with doing some trickery with the linker. I just wanted to be sure I am not missing an easier way.

[jimdempseyatthecove]

I've done this in the past.

What you might want to consider is to add a state flag to your replacement malloc/free and a means to set/reset it. When set, it calls the replacement malloc/free, when clear, it calls the "natural" malloc free.

Also, I suggest looking at TBB and how they can overload malloc/free/new/delete. While this will work within your app, you may have issues with libraries that you load that you cannot intercept the calls to malloc/free/new/delete/...

Also see: https://praba.wordpress.com/2009/01/14/detecting-memory-leaks-by-overloading-malloc/

Jim Dempsey

[KitturGanesh]

@Jim, adding a state flag on replacement malloc/free sounds like a workaround in the interim. BTW, I did pass on @djunglas and the developer will investigate and see what can be done in the subsequent versions as an enhancement, fyi.

_Kittur

[Marián__VooDooMan__M]

Greetings,

I will maybe need to do this in the future, but under MSVC + ICC. Could these Unix's flags be ported to ICC for Windows? Or is there some trick already for doing this?

best, vdm

[KitturGanesh]

Hi Marion, 

The scenario is the same with msvc + icc  in that there's no formal mechanism to redirect mem allocations as well. What @Jim is referring to is a user defined state flag setting dictating when the replacement or natural malloc is called (per @djunglas's requirement) which is a user implementation of the code.

_Kittur