Closed link for libraries using TBB offloading?

[Chris_G_]

Hot on the heels of my standalone success (Thanks Kevin) I tried to integrate a test of TBB offloading into our modular framework. In a nutshell, the user writes modules in C++ which get compiled into shared libraries, which are loaded at runtime into the framework when a particular module is required. For obvious reasons (the module writer is not able generally to re-link the exec) we do closed-link ("-Wl,--no-undefined" on linux) when making the shared library representing the module. However, despite my double-checking that the correct "-tbb" flags were used for both the source compile and library link, I get unresolved symbols at library link time:

$ /opt/intel/composer_xe_2013_sp1.2.144/bin/intel64/icpc  -fPIC -O3 -g -gdwarf-2 -fno-omit-frame-pointer -Werror -pedantic -std=c++11 -D_GLIBCXX_USE_NANOSLEEP  -Wall -Werror=return-type -Wextra -Wno-long-long -Winit-self -Woverloaded-virtual -tbb   -Wl,--no-undefined -shared -Wl,-soname,libtest_TestTBBOffload_module.so -o ../lib/libtest_TestTBBOffload_module.so CMakeFiles/test_TestTBBOffload_module.dir/TestTBBOffload_module.cc.o <libraries ...>

/home/greenc/work/cet-is/sources/art-hough/test/TestTBBOffload_module.cc:67: undefined reference to `__offload_target_acquire'
/home/greenc/work/cet-is/sources/art-hough/test/TestTBBOffload_module.cc:67: undefined reference to `__offload_offload1'

What am I missing?

Thanks for any pointers.

[Kevin_D_Intel]

Somehow the creation of the .so appears to be missing liboffload.so. The compiler driver may not be detecting the presence/use of offload constructs and thus perhaps not including it as expected. I'm guessing TestTBBOffload_module.cc.o has some offload directives. Those references might get resolved if you add the liboffload to your <libraries...> list yourself.

If there is any possibility of re-creating the undefined refs with a reproducer that would be great and we could better determine what is happening.

[Chris_G_]

Thank you. I'm not quite sure how the build system got past an earlier indicator of trouble, but I was able to solve the problem by carefully hiding our framework infrastructure from the mic-compiled code with "#ifndef __MIC__" clauses. I might actually have a substantive question to ask any day now. Thanks for your help.

[Kevin_D_Intel]

I have not been able to re-create the undefined references when creating an.so using the method you showed, so I still do not fully understand how that's happening. Without seeing how the "#ifndef __MIC__" is used I can't really comment but it seems unnecessary and I worry that it may cause other problems.

[Chris_G_]

Ours is quite the heavyweight framework from a non-domain-specific point of view, I suspect, but we try to conceptually (and often, physically) separate framework operations from algorithm- ("user") level code and inter-module communication objects. This reduces the amount of code we actually need to compile with the Intel compiler (14.0.2's C++2011 compliance isn't quite there for us yet, we have several bug reports still in engineering). The framework-level code and header inclusions may introduce link dependencies on code that the Intel Compiler didn't see and couldn't have produced offload code for, so it's better that it's hidden from the mic side, at least for now. When we're able to compile our whole suite with the Intel compiler set, we'll revisit the issue and see if we can't relax things. For now though, I've achieved our initial goals and we can now start exploring algorithm implementation aspects.

Thanks again for your help.

[Kevin_D_Intel]

You're welcome. Glad to hear about your achieving the initial goals and wish you well going forward.