Linker Problems with Nonstandard GCC Installation

[Marc_C_1]

In our CentOS 6.6 system we have a nonstandard gcc installation - let's say it's /opt/gcc49 - and the "standard" gcc installation as well, both with their own includes and libstdc++.

There's an option in icpc command, -cxxlib, which can be used to define the directory for C++ standard library. Using -cxxlib=/opt/gcc49 option helps with header files, but not with libstdc++ - even with this option it still takes /usr/lib/libstdc++.a instead of /opt/gcc49/lib/libstdc++.a

Passing the correct libstdc++ path to the command line helps as a workaround, but this hardly integrates with automated build tools.

[Shenghong_G_Intel]

It works for me, how did you know it is using the default libstdc++ instead of your non-standard installation? You may post your command line and how you get the conclusion that it is linking to wrong library.

See my testing below (Note: my default gcc is also a non-standard installation, which is gcc4.8.1, and I'll use -cxxlib to switch to gcc4.9.0). From the dry run (you can use -v, or -# or -dryrun options to get similiar information) results, the correct search path is used for C++ runtime libraries. Let me know your idea.

Thanks,

Shenghong

# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/opt/gcc/gcc-4.8.1/libexec/gcc/x86_64-unknown-linux-gnu/4.8.1/lto-wrapper
Target: x86_64-unknown-linux-gnu
Configured with: ../gcc-4.8.1/configure --prefix=/opt/gcc/gcc-4.8.1 --disable-nls
Thread model: posix
gcc version 4.8.1 (GCC)
# source /opt/intel/composer_xe_2015.2.164/bin/compilervars.sh intel64
# cat temp.cpp
#include <iostream>

int main() {
        std::cout<<"hello"<<std::endl;
        return 0;
}
# icc temp.cpp -static -v
icc version 15.0.2 (gcc version 4.8.1 compatibility)
/opt/intel/composer_xe_2015.2.164/bin/intel64/mcpcom    --lang=c++ -_g -mP3OPT_inline_alloca -D__ICC=1500 -D__INTEL_COMPILER=1500 -D__INTEL_COMPILER_UPDATE=2 -D__PTRDIFF_TYPE__=long "-D__SIZE_TYPE__=unsigned long" -D__WCHAR_TYPE__=int "-D__WINT_TYPE__=unsigned int" "-D__INTMAX_TYPE__=long int" "-D__UINTMAX_TYPE__=long unsigned int" -D__LONG_MAX__=9223372036854775807L -D__QMSPP_ -D__OPTIMIZE__ -D__NO_MATH_INLINES -D__NO_STRING_INLINES -D__GNUC_GNU_INLINE__ -D__GNUG__=4 -D__GNUC__=4 -D__GNUC_MINOR__=8 -D__GNUC_PATCHLEVEL__=1 -D__LP64__ -D_LP64 -D_GNU_SOURCE=1 -D__DEPRECATED=1 -D__GXX_WEAK__=1 -D__GXX_ABI_VERSION=1002 "-D__USER_LABEL_PREFIX__= " -D__REGISTER_PREFIX__= -D__INTEL_RTTI__ -D__EXCEPTIONS=1 -D__unix__ -D__unix -D__linux__ -D__linux -D__gnu_linux__ -B -Dunix -Dlinux "-_Asystem(unix)" -D__ELF__ -D__x86_64 -D__x86_64__ -D__amd64 -D__amd64__ "-_Acpu(x86_64)" "-_Amachine(x86_64)" -D_MT -D__INTEL_COMPILER_BUILD_DATE=20150121 -D__INTEL_OFFLOAD -D__i686 -D__i686__ -D__pentiumpro -D__pentiumpro__ -D__pentium4 -D__pentium4__ -D__tune_pentium4__ -D__SSE2__ -D__SSE2_MATH__ -D__SSE__ -D__SSE_MATH__ -D__MMX__ -_k -_8 -_l -_a -_b --gnu_version=481 -_W5 --gcc-extern-inline -p --bool -tused -x --multibyte_chars --array_section --simd --simd_func --offload_mode=1 --offload_target_names=gfx,GFX,mic,MIC --offload_unique_string=icc01010703573646VKCUC --bool -mP1OPT_version=15.0-intel64 -mGLOB_diag_file=/tmp/iccCibWLf.diag -mP1OPT_print_version=FALSE -mCG_use_gas_got_workaround=F -mP2OPT_align_option_used=TRUE -mGLOB_gcc_version=481 "-mGLOB_options_string=-static -v" -mGLOB_cxx_limited_range=FALSE -mCG_extend_parms=FALSE -mGLOB_compiler_bin_directory=/opt/intel/composer_xe_2015.2.164/bin/intel64 -mGLOB_as_output_backup_file_name=/tmp/iccyMoc19as_.s -mIPOPT_activate -mIPOPT_lite -mGLOB_em64t -mGLOB_instruction_tuning=0x0 -mGLOB_product_id_code=0x22006d91 -mCG_bnl_movbe=T -mGLOB_extended_instructions=0x8 -mP3OPT_use_mspp_call_convention -mP2OPT_subs_out_of_bound=FALSE -mP2OPT_disam_type_based_disam=2 -mP2OPT_disam_assume_ansi_c -mP2OPT_checked_disam_ansi_alias=TRUE -mGLOB_ansi_alias -mPGOPTI_value_profile_use=T -mGLOB_opt_report_use_source_name -mP2OPT_il0_array_sections=TRUE -mGLOB_offload_mode=1 -mP2OPT_offload_unique_var_string=icc01010703573646VKCUC -mP2OPT_hlo_level=2 -mP2OPT_hlo -mP2OPT_hpo_rtt_control=0 -mIPOPT_args_in_regs=0 -mP2OPT_disam_assume_nonstd_intent_in=FALSE -mGLOB_imf_mapping_library=/opt/intel/composer_xe_2015.2.164/bin/intel64/libiml_attr.so -mIPOPT_single_file_compile_and_link=TRUE -mPGOPTI_gen_threadsafe_level=0 -mIPOPT_obj_output_file_name=/tmp/iccCibWLf.o -mIPOPT_whole_archive_fixup_file_name=/tmp/iccwarchmsDVg4 -mGLOB_linker_version=2.20.51.0.2 -mGLOB_long_size_64 -mGLOB_routine_pointer_size_64 -mGLOB_driver_tempfile_name=/tmp/icctempfileTYkUo1 -mP3OPT_asm_target=P3OPT_ASM_TARGET_GAS -mGLOB_async_unwind_tables=TRUE -mGLOB_obj_output_file=/tmp/iccCibWLf.o -mGLOB_source_dialect=GLOB_SOURCE_DIALECT_C_PLUS_PLUS -mP1OPT_source_file_name=temp.cpp -mGLOB_eh_linux temp.cpp
#include "..." search starts here:
#include <...> search starts here:
 /opt/intel/composer_xe_2015.2.164/ipp/include
 /opt/intel/composer_xe_2015.2.164/mkl/include
 /opt/intel/composer_xe_2015.2.164/tbb/include
 /opt/intel/composer_xe_2015.2.164/ipp/include
 /opt/intel/composer_xe_2015.2.164/mkl/include
 /opt/intel/composer_xe_2015.2.164/tbb/include
 /opt/intel/composer_xe_2015.2.164/compiler/include/intel64
 /opt/intel/composer_xe_2015.2.164/compiler/include
 /opt/gcc/gcc-4.8.1/include/c++/4.8.1
 /opt/gcc/gcc-4.8.1/include/c++/4.8.1/x86_64-unknown-linux-gnu
 /opt/gcc/gcc-4.8.1/include/c++/4.8.1/backward
 /usr/local/include
 /opt/gcc/gcc-4.8.1/lib/gcc/x86_64-unknown-linux-gnu/4.8.1/include
 /opt/gcc/gcc-4.8.1/lib/gcc/x86_64-unknown-linux-gnu/4.8.1/include-fixed
 /opt/gcc/gcc-4.8.1/include/
 /usr/include
End of search list.
ld    /usr/lib/../lib64/crt1.o /usr/lib/../lib64/crti.o /opt/gcc/gcc-4.8.1/lib/gcc/x86_64-unknown-linux-gnu/4.8.1/crtbeginT.o --build-id -static -m elf_x86_64 -o a.out -L/opt/intel/composer_xe_2015.2.164/compiler/lib/intel64 -L/opt/intel/composer_xe_2015.2.164/ipp/../compiler/lib/intel64 -L/opt/intel/composer_xe_2015.2.164/ipp/lib/intel64 -L/opt/intel/composer_xe_2015.2.164/compiler/lib/intel64 -L/opt/intel/composer_xe_2015.2.164/mkl/lib/intel64 -L/opt/intel/composer_xe_2015.2.164/tbb/lib/intel64/gcc4.4 -L/opt/intel/composer_xe_2015.2.164/compiler/lib/intel64 -L/opt/intel/composer_xe_2015.2.164/ipp/../compiler/lib/intel64 -L/opt/intel/composer_xe_2015.2.164/ipp/lib/intel64 -L/opt/intel/composer_xe_2015.2.164/compiler/lib/intel64 -L/opt/intel/composer_xe_2015.2.164/mkl/lib/intel64 -L/opt/intel/composer_xe_2015.2.164/tbb/lib/intel64/gcc4.4 -L/opt/intel/composer_xe_2015.2.164/compiler/lib/intel64 -L/opt/gcc/gcc-4.8.1/lib/gcc/x86_64-unknown-linux-gnu/4.8.1/ -L/opt/gcc/gcc-4.8.1/lib/gcc/x86_64-unknown-linux-gnu/4.8.1/../../../../lib64 -L/opt/gcc/gcc-4.8.1/lib/gcc/x86_64-unknown-linux-gnu/4.8.1/../../../../lib64/ -L/lib/../lib64 -L/lib/../lib64/ -L/usr/lib/../lib64 -L/usr/lib/../lib64/ -L/opt/intel/composer_xe_2015.2.164/compiler/lib/intel64/ -L/opt/intel/composer_xe_2015.2.164/ipp/../compiler/lib/intel64/ -L/opt/intel/composer_xe_2015.2.164/ipp/lib/intel64/ -L/opt/intel/composer_xe_2015.2.164/mkl/lib/intel64/ -L/opt/intel/composer_xe_2015.2.164/tbb/lib/intel64/gcc4.4/ -L/opt/gcc/gcc-4.8.1/lib/gcc/x86_64-unknown-linux-gnu/4.8.1/../../../ -L/lib64 -L/lib/ -L/usr/lib64 -L/usr/lib /tmp/iccCibWLf.o -limf -lsvml -lirng -lm -lipgo -ldecimal -lstdc++ -lgcc -lgcc_eh -lirc -lsvml -lc -lgcc -lgcc_eh -lirc_s -ldl -lc /opt/gcc/gcc-4.8.1/lib/gcc/x86_64-unknown-linux-gnu/4.8.1/crtend.o /usr/lib/../lib64/crtn.o
# icc temp.cpp -cxxlib=/opt/gcc/gcc-4.9.0 -static -v
icc version 15.0.2 (gcc version 4.9.0 compatibility)
/opt/intel/composer_xe_2015.2.164/bin/intel64/mcpcom    --lang=c++ -_g -mP3OPT_inline_alloca -D__ICC=1500 -D__INTEL_COMPILER=1500 -D__INTEL_COMPILER_UPDATE=2 -D__PTRDIFF_TYPE__=long "-D__SIZE_TYPE__=unsigned long" -D__WCHAR_TYPE__=int "-D__WINT_TYPE__=unsigned int" "-D__INTMAX_TYPE__=long int" "-D__UINTMAX_TYPE__=long unsigned int" -D__LONG_MAX__=9223372036854775807L -D__QMSPP_ -D__OPTIMIZE__ -D__NO_MATH_INLINES -D__NO_STRING_INLINES -D__GNUC_GNU_INLINE__ -D__GNUG__=4 -D__GNUC__=4 -D__GNUC_MINOR__=9 -D__GNUC_PATCHLEVEL__=0 -D__LP64__ -D_LP64 -D_GNU_SOURCE=1 -D__DEPRECATED=1 -D__GXX_WEAK__=1 -D__GXX_ABI_VERSION=1002 "-D__USER_LABEL_PREFIX__= " -D__REGISTER_PREFIX__= -D__INTEL_RTTI__ -D__EXCEPTIONS=1 -D__unix__ -D__unix -D__linux__ -D__linux -D__gnu_linux__ -B -Dunix -Dlinux "-_Asystem(unix)" -D__ELF__ -D__x86_64 -D__x86_64__ -D__amd64 -D__amd64__ "-_Acpu(x86_64)" "-_Amachine(x86_64)" -D_MT -D__INTEL_COMPILER_BUILD_DATE=20150121 -D__INTEL_OFFLOAD -D__i686 -D__i686__ -D__pentiumpro -D__pentiumpro__ -D__pentium4 -D__pentium4__ -D__tune_pentium4__ -D__SSE2__ -D__SSE2_MATH__ -D__SSE__ -D__SSE_MATH__ -D__MMX__ -_k -_8 -_l -_a -_b --gnu_version=490 -_W5 --gcc-extern-inline -p --bool -tused -x --multibyte_chars --array_section --simd --simd_func --offload_mode=1 --offload_target_names=gfx,GFX,mic,MIC --offload_unique_string=icc0101650545992ogp5bG --bool -mP1OPT_version=15.0-intel64 -mGLOB_diag_file=/tmp/icconTH6h.diag -mP1OPT_print_version=FALSE -mCG_use_gas_got_workaround=F -mP2OPT_align_option_used=TRUE -mGLOB_gcc_version=490 "-mGLOB_options_string=-cxxlib=/opt/gcc/gcc-4.9.0 -static -v" -mGLOB_cxx_limited_range=FALSE -mCG_extend_parms=FALSE -mGLOB_compiler_bin_directory=/opt/intel/composer_xe_2015.2.164/bin/intel64 -mGLOB_as_output_backup_file_name=/tmp/iccs4tj7bas_.s -mIPOPT_activate -mIPOPT_lite -mGLOB_em64t -mGLOB_instruction_tuning=0x0 -mGLOB_product_id_code=0x22006d91 -mCG_bnl_movbe=T -mGLOB_extended_instructions=0x8 -mP3OPT_use_mspp_call_convention -mP2OPT_subs_out_of_bound=FALSE -mP2OPT_disam_type_based_disam=2 -mP2OPT_disam_assume_ansi_c -mP2OPT_checked_disam_ansi_alias=TRUE -mGLOB_ansi_alias -mPGOPTI_value_profile_use=T -mGLOB_opt_report_use_source_name -mP2OPT_il0_array_sections=TRUE -mGLOB_offload_mode=1 -mP2OPT_offload_unique_var_string=icc0101650545992ogp5bG -mP2OPT_hlo_level=2 -mP2OPT_hlo -mP2OPT_hpo_rtt_control=0 -mIPOPT_args_in_regs=0 -mP2OPT_disam_assume_nonstd_intent_in=FALSE -mGLOB_imf_mapping_library=/opt/intel/composer_xe_2015.2.164/bin/intel64/libiml_attr.so -mIPOPT_single_file_compile_and_link=TRUE -mPGOPTI_gen_threadsafe_level=0 -mIPOPT_obj_output_file_name=/tmp/icconTH6h.o -mIPOPT_whole_archive_fixup_file_name=/tmp/iccwarchIB3o85 -mGLOB_linker_version=2.20.51.0.2 -mGLOB_long_size_64 -mGLOB_routine_pointer_size_64 -mGLOB_driver_tempfile_name=/tmp/icctempfile6Ux48x -mP3OPT_asm_target=P3OPT_ASM_TARGET_GAS -mGLOB_async_unwind_tables=TRUE -mGLOB_obj_output_file=/tmp/icconTH6h.o -mGLOB_source_dialect=GLOB_SOURCE_DIALECT_C_PLUS_PLUS -mP1OPT_source_file_name=temp.cpp -mGLOB_eh_linux temp.cpp
#include "..." search starts here:
#include <...> search starts here:
 /opt/intel/composer_xe_2015.2.164/ipp/include
 /opt/intel/composer_xe_2015.2.164/mkl/include
 /opt/intel/composer_xe_2015.2.164/tbb/include
 /opt/intel/composer_xe_2015.2.164/ipp/include
 /opt/intel/composer_xe_2015.2.164/mkl/include
 /opt/intel/composer_xe_2015.2.164/tbb/include
 /opt/intel/composer_xe_2015.2.164/compiler/include/intel64
 /opt/intel/composer_xe_2015.2.164/compiler/include
 /opt/gcc/gcc-4.9.0/include/c++/4.9.0
 /opt/gcc/gcc-4.9.0/include/c++/4.9.0/x86_64-unknown-linux-gnu
 /opt/gcc/gcc-4.9.0/include/c++/4.9.0/backward
 /usr/local/include
 /opt/gcc/gcc-4.9.0/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/include
 /opt/gcc/gcc-4.9.0/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/include-fixed
 /opt/gcc/gcc-4.9.0/include/
 /usr/include
End of search list.
ld    /usr/lib/../lib64/crt1.o /usr/lib/../lib64/crti.o /opt/gcc/gcc-4.9.0/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/crtbeginT.o --build-id -static -m elf_x86_64 -o a.out -L/opt/intel/composer_xe_2015.2.164/compiler/lib/intel64 -L/opt/intel/composer_xe_2015.2.164/ipp/../compiler/lib/intel64 -L/opt/intel/composer_xe_2015.2.164/ipp/lib/intel64 -L/opt/intel/composer_xe_2015.2.164/compiler/lib/intel64 -L/opt/intel/composer_xe_2015.2.164/mkl/lib/intel64 -L/opt/intel/composer_xe_2015.2.164/tbb/lib/intel64/gcc4.4 -L/opt/intel/composer_xe_2015.2.164/compiler/lib/intel64 -L/opt/intel/composer_xe_2015.2.164/ipp/../compiler/lib/intel64 -L/opt/intel/composer_xe_2015.2.164/ipp/lib/intel64 -L/opt/intel/composer_xe_2015.2.164/compiler/lib/intel64 -L/opt/intel/composer_xe_2015.2.164/mkl/lib/intel64 -L/opt/intel/composer_xe_2015.2.164/tbb/lib/intel64/gcc4.4 -L/opt/intel/composer_xe_2015.2.164/compiler/lib/intel64 -L/opt/gcc/gcc-4.9.0/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/ -L/opt/gcc/gcc-4.9.0/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../lib64 -L/opt/gcc/gcc-4.9.0/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../lib64/ -L/lib/../lib64 -L/lib/../lib64/ -L/usr/lib/../lib64 -L/usr/lib/../lib64/ -L/opt/intel/composer_xe_2015.2.164/compiler/lib/intel64/ -L/opt/intel/composer_xe_2015.2.164/ipp/../compiler/lib/intel64/ -L/opt/intel/composer_xe_2015.2.164/ipp/lib/intel64/ -L/opt/intel/composer_xe_2015.2.164/mkl/lib/intel64/ -L/opt/intel/composer_xe_2015.2.164/tbb/lib/intel64/gcc4.4/ -L/opt/gcc/gcc-4.9.0/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../ -L/lib64 -L/lib/ -L/usr/lib64 -L/usr/lib /tmp/icconTH6h.o -limf -lsvml -lirng -lm -lipgo -ldecimal -lstdc++ -lgcc -lgcc_eh -lirc -lsvml -lc -lgcc -lgcc_eh -lirc_s -ldl -lc /opt/gcc/gcc-4.9.0/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/crtend.o /usr/lib/../lib64/crtn.o
#

[Marc_C_1]

I don't know that it takes exactly /usr/lib/libstdc++a, but it causes linker errors that report symbols from libstdc++ - the same if I'd use "normal" gcc in this situation.

This doesn't happen if I do either of:
   - use /opt/gcc49/bin/g++-4.9 to link it
   - add /opt/gcc49/lib/libstdc++.a at the end of 'icpc'-call command line

The interesting part of the command line:

-lQtTest -lQtScriptTools -lQtScript -lQtSvg -lQtSql -lQtXmlPatterns -lQtXml -lQtOpenGL -lQtGui -lQtNetwork -lQtCore -lGL -lpthread -Bdynamic -Bstatic -limf -lsvml -lirng -Bdynamic -lm -Bstatic -lipgo -ldecimal --as-needed -Bdynamic -lcilkrts --no-as-needed -lstdc++ -lgcc -lgcc_s -Bstatic -lirc -lsvml -Bdynamic -lc -lgcc -lgcc_s -Bstatic -lirc_s -Bdynamic -ldl -lc /usr/lib/gcc/x86_64-redhat-linux/4.4.7/crtend.o /usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../lib64/crtn.o

You vcan see explicit "-lstdc++" used here - it happens independently on having -cxxlib=/opt/gcc49 in the options or not.

[TimP]

Currently supported versions of icpc should work with g++ 4.9.  I think icpc 13.x will not work with the newer g++.

[Shenghong_G_Intel]

From the command line, it is still using gcc 4.4.7 (the crtn.o). I do not think your -cxxlib is taking effect....And also, why -lstdc++ should be provided explicitly. I assume you are building with a free-standing environment or something like that. Or maybe your build system is complex and you add the -cxxlib in a wrong place?

Is it possible to post your project to me and I'll check it for you?

Thanks,

Shenghong

[Shenghong_G_Intel]

Also, what is your version of ICC? As Tim mentioned, old ICC may not support gcc4.9 and I am not sure whether that affects -cxxlib...

Thanks,

Shenghong

[Marc_C_1]

Hi Shengong,

This is CentOS 6.6, which contains gcc 4.4.7 as the latest provided version. We have a software that requires C++11 and because of that we need some more recent version, and that's why we have our "homecooked" installation of gcc 4.9 (which's "SDK" part is required by Intel compiler). This comes with its own libstdc++ - and when I'm compiling the same software using our /opt/gcc49/bin/g++-4.9 there are no linkage problems - it selects correct libstdc++.

As I have tried the command line with -v, the icpc command resolves finally to call ld where the standard C++ library flag is passed as "-lstdc++", which simply leads to using the version of libstdc++ whichever is currently defined by following the /etc/ld.so.conf settings and environment variables. This may be an expected behavior, as long as you are using the "system standard" gcc installation. But I guess the "-cxxlib" option has been added to point to ANOTHER gcc "SDK" - and "SDK" consists of set of include files and libraries. So if the compiler is going to use include files from /opt/gcc49/include, then it should also take libstdc++ from /opt/gcc49/lib (or /opt/gcc49/lib64 in our case), as gcc does - otherwise it picks up two mutually incompatible software elements.

In short: if icpc is passed -cxxlib=/opt/gcc49, then it should pass "/opt/gcc49/lib(64)/libstdc++(.a or .so)" to the ld command, NOT -lstdc++.

Thanks,
Marc