Compiling Taglib for Android using CMake yields "mulltiple definition of symbol `__i686.get_pc_thunk.bx'"

[Michał_K_1]

Hi!
Our Android application uses Taglib compiled as a shared library. It has been successfully compiled using GNU toolchain from NDK (and this version is available on the market). Now we are trying to switch to the Intel compiler. However, compilation using CMake yields the following output:

[ 98%] Built target tag
Linking CXX shared library libtag_c.so
Error: multiple definition of symbol `__i686.get_pc_thunk.bx'
Error: multiple definition of symbol `__i686.get_pc_thunk.bx'
Error: multiple definition of symbol `__i686.get_pc_thunk.bx'
make[2]: *** [bindings/c/libtag_c.so.0.0.0] Error 1
make[1]: *** [bindings/c/CMakeFiles/tag_c.dir/all] Error 2
make: *** [all] Error 2

I know that __i686.get_pc_thunk.bx is a function related to position independent code, but it doesn't get me anywhere. I've read that removing all compiler options might help and so I removed them (at least these which I could find, CMake uses scripts to generate scripts and "-show" option is not supported by icpc, so I'm not sure everything is clean). Nothing seems to make a difference.

libtag.a is produced; libtag_c.so is not. We later use libtag.a in Eclipse to generate a different so - it fails with exactly the same error, so the problem emerges at the linking stage. I wrote a simple c++ file with one function (not reliant on stl or anything) and compiled it into a shared library using icpc directly from command line - it worked. We also have the Intel compiler hooked to Eclipse - compilation of c++ files (without any dependency on libtag.a) into an so works without issue.

BTW >> we've solved (using StackOverflow) the issue with templates which comes up first when attempting to compile Taglib using icpc.

Other information:
compiler version is: icpc (ICC) 13.0.1 20130722

NDK version is: r8e
icc-toolchain.cmake:

set(_toolchain_C_command   "icc" )
set(_toolchain_CXX_command "icpc")
set(_toolchain_have_isystem 1    )

include_directories("/home/lurker/Downloads/android-ndk-r8e/sources/cxx-stl/gnu-libstdc++/4.6/include/")
include_directories("/home/lurker/Downloads/android-ndk-r8e/sources/cxx-stl/gnu-libstdc++/4.6/libs/x86/include/")

include("${CMAKE_CURRENT_LIST_DIR}/simple-toolchain.cmake")

simple-toolchain.cmake:

foreach(lang IN ITEMS C CXX)

    find_program(TOOLCHAIN_${_toolchain_${lang}_command}_BINARY
                 "${_toolchain_${lang}_command}")
    mark_as_advanced(TOOLCHAIN_${_toolchain_${lang}_command}_BINARY)

    if(NOT TOOLCHAIN_${_toolchain_${lang}_command}_BINARY)
        message(FATAL_ERROR "${_toolchain_${lang}_command} not found")
    endif()

    set(CMAKE_${lang}_COMPILER "${TOOLCHAIN_${_toolchain_${lang}_command}_BINARY}")

    unset(_toolchain_${lang}_command)

endforeach()

if(_toolchain_have_isystem)
    # Tell cmake that the compiler supports -isystem
    set(CMAKE_INCLUDE_SYSTEM_FLAG_C "-isystem ")
    set(CMAKE_INCLUDE_SYSTEM_FLAG_CXX "-isystem ")
endif()
unset(_toolchain_have_isystem)

build command:
cmake -G "Unix Makefiles"  -DCMAKE_TOOLCHAIN_FILE=$CMAKE_TOOLCHAIN -DCMAKE_MAKE_PROGRAM="make" -DBUILD_SHARED_LIBS=OFF -DCMAKE_BUILD_TYPE="Release" -DANDROID_USE_STLPORT=FALSE -DANDROID_ABI=x86
make -j4

[Michał_K_1]

Small update -> I compiled everything manually (just to be sure that I have total control), using (repeated in every dir, paths are prefixed with -I):

icpc -c *.cpp $ANDROID_INCLUDE_PATH $TAGLIB_INCLUDE_PATH

Everything went fine. But then I tried to link it using:

icpc -o libtag.so -shared -fpic LIST_OF_FILES_WRITTE_BY_HAND

Same error:

Error: multiple definition of symbol `__i686.get_pc_thunk.bx'
Error: multiple definition of symbol `__i686.get_pc_thunk.bx'
Error: multiple definition of symbol `__i686.get_pc_thunk.bx'

Could you provide some insight into the cause this issue?

[Alexander_W_Intel]

Hi,

just to confirm: You are using the Intel® C++ Compiler forAndroid?

Please try to link without the -fpic option. The Android version of the Compiler should link the code position independend by default. 

Please note that you should also specify the option -static-intel for linking. 

Let me know if that helped :-)
Alex 

[Michał_K_1]

Alexander Weggerle (Intel) wrote:

just to confirm: You are using the Intel® C++ Compiler forAndroid?

Please try to link without the -fpic option. The Android version of the Compiler should link the code position independend by default. 

Please note that you should also specify the option -static-intel for linking. 

Hi,

Thank you for your answer. I removed the -fpic option and it didn't help. Then I added -static-intel. Errors are still the same. Then - out of desperation - I tried xild  directly with -shared -o libtaglib.so libtag.a and other compiled object files. It said it doesn't recognize -static-intel and of course failed.

Yes, I am using Intel Compiler for Android, unpacked and installed from l_CCAndroid_p_13.0.1.020.tgz file, downloaded from Intel's website.

The issue is still open - do you have any other ideas what I am doing wrong?

[MalReddy_Y_Intel · Accepted solution]

Hi,

You can trace the files used by linker and even search the definition (multiple) of symbols where linker is finding using below commands.

#icpc -o libtag.so -shared -fpic -Wl,-t LIST_OF_FILES_WRITTE_BY_HAND

#icpc -o libtag.so -shared -fpic -Wl,-y,__i686.get_pc_thunk.bx LIST_OF_FILES_WRITTE_BY_HAND

Hope this will help you to find the cause.

Regards,

Reddy

[Alexander_W_Intel]

Hi,

can you send a small reproducer for this issue? You can use the "Send Author a message" link to send a private message if you don't want to share your reproducer publically. 

Thanks,
Alex 

[Michał_K_1]

Y. R. (Intel) wrote:

Hi,

You can trace the files used by linker and even search the definition (multiple) of symbols where linker is finding using below commands.

#icpc -o libtag.so -shared -fpic -Wl,-t LIST_OF_FILES_WRITTE_BY_HAND

#icpc -o libtag.so -shared -fpic -Wl,-y,__i686.get_pc_thunk.bx LIST_OF_FILES_WRITTE_BY_HAND

Hope this will help you to find the cause.

Hi,

The issue is still not entirely solved, but the options you provided really did help in narrowing down the source and moving the compilation a little bit further. It seems that there is library libirc_pic.a which has multiple object files defining the same symbol __i686.get_pc_thunk.bx

Part of the output of nm utility ran on libirc_pic.a:

intel_sse2_memset.o:
         U _GLOBAL_OFFSET_TABLE_
00000000 T __i686.get_pc_thunk.bx
00000000 T __intel_sse2_memset
         U __libirc_data_cache_size
         U __libirc_largest_cache_size
        
intel_ssse3_memcpy.o:
         U _GLOBAL_OFFSET_TABLE_
00000000 T __i686.get_pc_thunk.bx
00000000 T __intel_ssse3_memcpy
         U __libirc_data_cache_size_half
         U __libirc_largest_cache_size_half

After some tweaks, the output of icpc -Wl,-t is:

Searching /home/lurker/intel/CCAndroid13.0.1.020/lib/libirc_pic.a:
Found __intel_sse2_memset
        Referenced in toolkit/tbytevector.o
        Loaded libirc_pic.a(intel_sse2_memset.o)

Found __intel_ssse3_memcpy
        Referenced in toolkit/tbytevector.o
        Loaded libirc_pic.a(intel_ssse3_memcpy.o)

And output of icpc -Wl,-y,__i686.get_pc_thunk.bx is:

/home/lurker/intel/CCAndroid13.0.1.020/lib/libirc_pic.a(intel_sse2_memset.o): definition of __i686.get_pc_thunk.bx
/home/lurker/intel/CCAndroid13.0.1.020/lib/libirc_pic.a(intel_ssse3_memcpy.o): definition of __i686.get_pc_thunk.bx

But the shared library is compiled! By "some tweaks" I mean adding "-xSSSE3_ATOM" flag at the compilation stage. I did it because when I compiled the code without it, more objects from libirc_pic.a were loaded (like operations for sse2, sse3, generic and all...) and so more definitions of __i686.get_pc.thunk.bx were present. It seems that setting the architecture explicitly reduced the number of used functions and so object files and now there are no collisions.... Thank you very much Reddy!

The current issue is that when I run the application with the compiled so, I get UnsatisfiedLinkError when loading the lib:

E/dalvikvm(11186): dlopen("/data/app-lib/MY_APPLICATION/libtaglibbinding.so") failed: Cannot load library: soinfo_relocate(linker.cpp:975): cannot locate symbol "_ZSt18_Rb_tree_decrementPKSt18_Rb_tree_node_base" referenced by "libtaglibbinding.so"...

I'm working on it right now. If anyone could help, I would appreciate it, but it seems that it's a different issue. What I'm suspecting now is that although I use -static-intel, the linker seems to search for functions in:

/home/lurker/Downloads/android-ndk-r8e/platforms/android-9/arch-x86/usr/lib/libstdc++.so

and

/home/lurker/Downloads/android-ndk-r8e/toolchains/x86-4.6/prebuilt/linux-x86/bin/../lib/gcc/i686-linux-android/4.6//libgcc.a

Is this a valid behaviour?
Best regards,
Michal

[Michał_K_1]

Michał K. wrote:

Quote:

The current issue is that when I run the application with the compiled so, I get UnsatisfiedLinkError when loading the lib:

E/dalvikvm(11186): dlopen("/data/app-lib/MY_APPLICATION/libtaglibbinding.so") failed: Cannot load library: soinfo_relocate(linker.cpp:975): cannot locate symbol "_ZSt18_Rb_tree_decrementPKSt18_Rb_tree_node_base" referenced by "libtaglibbinding.so"...

I'm working on it right now. If anyone could help, I would appreciate it, but it seems that it's a different issue.

Hi,

That one is now solved by adding stl explicitly using:

icpc -shared -fpic  -o libtaglibbinding.so *.o taglib/x86/libtag.a /home/lurker/Downloads/android-ndk-r8e/sources/cxx-stl/gnu-libstdc++/4.6/libs/x86/libgnustl_static.a

But now the application cannot load libsvml.so in runtime. Adding "/home/lurker/intel/CCAndroid13.0.1.020/lib/libsvml.a" to linking doesn't work. "-lsvml" does nothing. Adding "-static" or "-static-intel" with "-lsvml" doesn't change a thing (the output file is the same).

Best regards,

Michal Kurowski

[MalReddy_Y_Intel]

Hi,

What is the error that you are getting with reference to loading of libsvml.so ?

Please send us appropriate error message with the commandline that you are trying, So that people will suggest the right method of linking.

Reddy

[Michał_K_1]

Y. R. (Intel) wrote:

What is the error that you are getting with reference to loading of libsvml.so ?

Please send us appropriate error message with the commandline that you are trying, So that people will suggest the right method of linking.

Hi!

The error I was getting simply said that libsvml.so couldn't be loaded. However, the weekend seems to have cleared my mind, I read the help for icpc and our previous building scripts once again and got everything working properly.

My current (working) commandline is:

icpc -shared -fpic -static-intel -static-libstdc++  -o libtaglibbinding.so *.o taglib/x86/libtag.a libgnustl_static.a -landroid -llog

Now I will play with the optimization options. Thank you all for your help. The issue is finally solved. The real blocking issue was solved by using commands proposed by Reddy ->

#icpc -o libtag.so -shared -fpic -Wl,-t LIST_OF_FILES_WRITTE_BY_HAND

#icpc -o libtag.so -shared -fpic -Wl,-y,__i686.get_pc_thunk.bx LIST_OF_FILES_WRITTE_BY_HAND

The rest is... my negligence and my CLI getting rusty. Thanks again!
Best regards,
Michal