Duplicate symbols when linking object files compiled with -ipo

[djunglas]

Hi there,

I use icc 12.1 SP1.11.339 on 64bit Linux and get duplicate symbol errors for symbols like .L_2__STRING.15.6 when trying to link object files that have been compiled with the -ipo option.

What I am trying to do is the following:

Our code contains hundreds of source files. For several reasons we cannot perform IPO on all of the files simultaneously. What we instead want to do is to package the files into a few "modules", perform IPO on each module, create one object file per module, and then link with these module object files (eventually the module object files will go into a static library that is shipped to the customer). So far everything but linking the application works well. Here is what I am doing:

First I compile each source file individually:
[bash]icc -ipo -vec-report0 -O -fPIC -fno-strict-aliasing -fvisibility=hidden -Wall -Wmissing-prototypes -Wmissing-declarations -Wshadow -fno-builtin-strlen -fno-builtin-strcat -fno-builtin-strcmp -fno-builtin-strcpy -fno-builtin-strncat -fno-builtin-strncmp -fno-builtin-strrchr  -c -o file.o file.c[/bash]

Next I perform an incremental link to combine multiple object files into one module object file
[bash]
icc -diag-disable 1419 -w1 -Wcheck -vec-report0 -O -fPIC -fno-strict-aliasing -fvisibility=hidden -Wall -Wmissing-prototypes -Wmissing-declarations -Wshadow -fno-builtin-strlen -fno-builtin-strcat -fno-builtin-strcmp -fno-builtin-strcpy -fno-builtin-strncat -fno-builtin-strncmp -fno-builtin-strrchr -Wl,-i -nostdlib -o .module1.o file1.o ... filen.o
ipo: remark #11000: performing multi-file optimizations
ipo-1: remark #11006: generating object file /tmp/ipo_iccAP7F4c1.o
ipo-2: remark #11006: generating object file /tmp/ipo_iccAP7F4c2.o
ipo-3: remark #11006: generating object file /tmp/ipo_iccAP7F4c3.o
ipo-4: remark #11006: generating object file /tmp/ipo_iccAP7F4c4.o
[/bash]

Finally, I link an application with the module object files (note that the main?.o files were also compiled with -ipo):
[bash]icc -Wl,--sort-section=name -o program  main1.o ... mainn.o module1.o ... modulen.o -static-intel -Wl,--start-group /path/to/icc/12.1/composer_xe_2011_sp1.11.339/compiler/lib/intel64/libirc.a /path/to/icc/12.1/composer_xe_2011_sp1.11.339/mkl/lib/intel64/libmkl_intel_lp64.a /path/to/icc/12.1/composer_xe_2011_sp1.11.339/mkl/lib/intel64/libmkl_sequential.a /path/to/icc/12.1/composer_xe_2011_sp1.11.339/mkl/lib/intel64/libmkl_core.a -Wl,--end-group -lpthread -lm -ldl -rdynamic
icc: warning #10237: -lcilkrts linked in dynamically, static library not available
ipo: remark #11000: performing multi-file optimizations
ipo-1: remark #11006: generating object file /tmp/ipo_iccCzLffg1.o
ipo-2: remark #11006: generating object file /tmp/ipo_iccCzLffg2.o
ipo-3: remark #11006: generating object file /tmp/ipo_iccCzLffg3.o
ipo-4: remark #11006: generating object file /tmp/ipo_iccCzLffg4.o
module1.o:(.rodata+0x1a20): multiple definition of `.L_2__STRING.15.6'
/tmp/ipo_iccCzLffg2.o:(.rodata+0x5138): first defined here
ld: Warning: size of symbol `.L_2__STRING.15.6' changed from 10 in /tmp/ipo_iccCzLffg2.o to 17 in .OPT/module1.o
... (many more duplicated symbols with similar names)
[/bash]

The value of symbol .L_2__STRING.15.6 seems to be a string constant that is defined in our source code. But why is it defined multiple times? Can anybody see what I am doing wrong? Everything works fine if I don't use -ipo to compile the individual object files.

Thanks a lot

Daniel

[djunglas]

So far I could not find a way to avoid the duplicate symbols. However, I managed to work around the problem by renaming the offending symbols using 'objcopy --redefine-syms'. This allowed me to at least link the binary. I hope it will not cause trouble at runtime ...

[SergeyKostrov]

>>The value of symbol .L_2__STRING.15.6 seems to be a string constant that is defined in our source code. But why is it >>defined multiple times? Can anybody see what I am doing wrong? Did you define that string constant in a header file or in a C/C++ file? If it is defined in some header file it could be the source of the linker error.

[SergeyKostrov]

Please read it carefully. >>... >>/tmp/ipo_iccCzLffg2.o:(.rodata+0x5138): first defined here >> >>10 ld: Warning: size of symbol `.L_2__STRING.15.6' changed from 10 in /tmp/ipo_iccCzLffg2.o to 17 in .OPT/module1.o >> >>11 ... (many more duplicated symbols with similar names) >>... I see that the string constant is declared at least twice and there is some inconsistency with its declaration.

[djunglas]

Sergey Kostrov wrote:

>>The value of symbol .L_2__STRING.15.6 seems to be a string constant that is defined in our source code. But why is it
>>defined multiple times? Can anybody see what I am doing wrong?

Did you define that string constant in a header file or in a C/C++ file? If it is defined in some header file it could be the source of the linker error.

There are actually many duplicate symbols. Some are for string constants that are defined in header files, some for string constants that are defined in C files. Note that all these constants are either defined as string literals or macros, that is, they are all of the form [cpp] printf ("Some string constant\n"); #define STRING_CONSTANT "Another string constant" [/cpp] so our source code does not explicitly create symbols for those constants.

[djunglas]

Sergey Kostrov wrote:

Please read it carefully.

>>...
>>/tmp/ipo_iccCzLffg2.o:(.rodata+0x5138): first defined here
>>
>>10 ld: Warning: size of symbol `.L_2__STRING.15.6' changed from 10 in /tmp/ipo_iccCzLffg2.o to 17 in .OPT/module1.o
>>
>>11 ... (many more duplicated symbols with similar names)
>>...

I see that the string constant is declared at least twice and there is some inconsistency with its declaration.

Yes, that is exactly the issue. The problem is that I do not define these .L_2__STRING.15.6 symbols. It looks like icc automatically generates these names and assigns them to string constants? And it looks as if if may choose the same auto-generated symbol names for different translation units. When I try to link those translation units then I will of course run into trouble.

[djunglas]

I have an interesting update: I just found that adding -ipo1 to the incremental link step seems to fix the problem as well -- previously I was not using any -ipo argument in that step. Linking the "modules" seems to take much longer but at least there are no more issues with duplicate symbols.

[SergeyKostrov]

Thanks for the note about your solution based on using the '-ipo1' argument.