Community
cancel
Showing results for 
Search instead for 
Did you mean: 
avayasheik
Beginner
234 Views

icc link error

Hi;
We are evaluating the Intel Composer on RedHat Linux as a possible compiler. I am using the -ipo optimization flag during compiling/linking. The entire system seems to compile and link okay with the exception of a single module.

I am getting the following error during link time....I tried increasing the number of optimization files to 8 (-ipo8) but no luck.
Please note that using gcc everything compile and links okay. I would appreciate some guidance.
Thanks in advance.
icc version is 12.0.3




ipo: remark #11000: performing multi-file optimizations
ipo-1: remark #11006: generating object file /tmp/ipo_icpcIlF9nY1.o
Engine/src/SEHDiamC.cpp(261): (col. 1) remark: _ZN14SEHDiamCStatus11isConnectedEv has been targeted for automatic cpu dispatch.
ipo-2: remark #11006: generating object file /tmp/ipo_icpcIlF9nY2.o
Store/StoreAPI/src/StoreDBManager.cpp(1406): (col. 1) remark: _ZN15StoreDBManager27importMMFDirectoryByExtLinkEPc has been targeted for automatic cpu dispatch.
ipo-3: remark #11006: generating object file /tmp/ipo_icpcIlF9nY3.o
ipo-4: remark #11006: generating object file /tmp/ipo_icpcIlF9nY4.o
../../Controller/Adapter/src/Engine.cpp(559): (col. 1) remark: _ZN8MAEngine17executeAcquireReqEP11AppSyncLineP13AcquireSvcReq has been targeted for automatic cpu dispatch.
../../Controller/Adapter/src/Engine.cpp(580): (col. 1) remark: _ZN8MAEngine22executeAllocPoolSvcReqEP11AppSyncLineP15AllocPoolSvcReq has been targeted for automatic cpu dispatch.
Controller/MAdapter/src/MConfigAdapter.cpp(120): (col. 1) remark: _Z15bsibb_getLocaleP7session has been targeted for automatic cpu dispatch.
../../Controller/Adapter/src/MAMxAdapter.cpp(91): (col. 1) remark: _ZN11MAMxAdapter10initializeEv has been targeted for automatic cpu dispatch.
(0): internal error: backend signals

ipo-4: error #11005: multi-object compilation 4 returned error status 4
icpc: error #10014: problem during multi-file optimization compilation (code 4)
...failed intel-linux.link .....

0 Kudos
12 Replies
TimP
Black Belt
234 Views

Well, gcc doesn't have a multiple CPU architecture path option, but you don't give enough information to guess your gcc and icc options. Internal errors, of course, are always sufficient reason for posting a problem report with reproducer on your premier.intel.com account.
gcc option -lto has similar intent to icc -ipo (but may not be supported by a gcc provided by Red Hat prior to 6.0)
Om_S_Intel
Employee
234 Views

You may use -ipo- compiler option to disable IPO optimization.
aazue
New Contributor I
234 Views

Hi
This problem occurred when several functions have same name in multiple objects and is not correctly regrouped
resolved by compiler.
problem could be more complex to find with used sided C++ the class (member)
use gcc with flag -fwhole-program probably you discover more information where is the problem
also you can isolate incriminated with -fuse-linker-plugin.
But sometime is persisting complex problem that remains in darkness

about:(Well, gcc doesn't have a multiple CPU architecture path option)

Inform more with GNU compiler actualized...

-fwhole-program
Assume that the current compilation unit represents the whole program being compiled. All public functions and variables with the exception of main and those merged by attribute externally_visible become static functions and in effect are optimized more aggressively by interprocedural optimizers. If gold is used as the linker plugin, externally_visible attributes are automatically added to functions (not variable yet due to a current gold issue) that are accessed outside of LTO objects according to resolution file produced by gold. For other linkers that cannot generate resolution file, explicit externally_visible attributes are still necessary. While this option is equivalent to proper use of the static keyword for programs consisting of a single file, in combination with option -flto this flag can be used to compile many smaller scale programs since the functions and variables become local for the whole combined compilation unit, not for the single source file itself.

This option implies -fwhole-file for Fortran programs.
-flto[=n]
This option runs the standard link-time optimizer. When invoked with source code, it generates GIMPLE (one of GCC's internal representations) and writes it to special ELF sections in the object file. When the object files are linked together, all the function bodies are read from these ELF sections and instantiated as if they had been part of the same translation unit.

To use the link-timer optimizer, -flto needs to be specified at compile time and during the final link. For example,

gcc -c -O2 -flto foo.c
gcc -c -O2 -flto bar.c
gcc -o myprog -flto -O2 foo.o bar.o


The first two invocations to GCC will save a bytecode representation of GIMPLE into special ELF sections inside foo.o and bar.o. The final invocation will read the GIMPLE bytecode from foo.o and bar.o, merge the two files into a single internal image, and compile the result as usual. Since both foo.o and bar.o are merged into a single image, this causes all the inter-procedural analyses and optimizations in GCC to work across the two files as if they were a single one. This means, for example, that the inliner will be able to inline functions in bar.o into functions in foo.o and vice-versa.

Another (simpler) way to enable link-time optimization is,

gcc -o myprog -flto -O2 foo.c bar.c


The above will generate bytecode for foo.c and bar.c, merge them together into a single GIMPLE representation and optimize them as usual to produce myprog.

The only important thing to keep in mind is that to enable link-time optimizations the -flto flag needs to be passed to both the compile and the link commands.

To make whole program optimization effective, it is necessary to make certain whole program assumptions. The compiler needs to know what functions and variables can be accessed by libraries and runtime outside of the link time optimized unit. When supported by the linker, the linker plugin (see -fuse-linker-plugin) passes to the compiler information about used and externally visible symbols. When the linker plugin is not available, -fwhole-program should be used to allow the compiler to make these assumptions, which will lead to more aggressive optimization decisions.

Note that when a file is compiled with -flto, the generated object file will be larger than a regular object file because it will contain GIMPLE bytecodes and the usual final code. This means that object files with LTO information can be linked as a normal object file. So, in the previous example, if the final link is done with

gcc -o myprog foo.o bar.o


The only difference will be that no inter-procedural optimizations will be applied to produce myprog. The two object files foo.o and bar.o will be simply sent to the regular linker.

Additionally, the optimization flags used to compile individual files are not necessarily related to those used at link-time. For instance,

gcc -c -O0 -flto foo.c
gcc -c -O0 -flto bar.c
gcc -o myprog -flto -O3 foo.o bar.o


This will produce individual object files with unoptimized assembler code, but the resulting binary myprog will be optimized at -O3. Now, if the final binary is generated without -flto, then myprog will not be optimized.

When producing the final binary with -flto, GCC will only apply link-time optimizations to those files that contain bytecode. Therefore, you can mix and match object files and libraries with GIMPLE bytecodes and final object code. GCC will automatically select which files to optimize in LTO mode and which files to link without further processing.

There are some code generation flags that GCC will preserve when generating bytecodes, as they need to be used during the final link stage. Currently, the following options are saved into the GIMPLE bytecode files: -fPIC, -fcommon and all the -m target flags.

At link time, these options are read-in and reapplied. Note that the current implementation makes no attempt at recognizing conflicting values for these options. If two or more files have a conflicting value (e.g., one file is compiled with -fPIC and another isn't), the compiler will simply use the last value read from the bytecode files. It is recommended, then, that all the files participating in the same link be compiled with the same options.

Another feature of LTO is that it is possible to apply interprocedural optimizations on files written in different languages. This requires some support in the language front end. Currently, the C, C++ and Fortran front ends are capable of emitting GIMPLE bytecodes, so something like this should work

gcc -c -flto foo.c
g++ -c -flto bar.cc
gfortran -c -flto baz.f90
g++ -o myprog -flto -O3 foo.o bar.o baz.o -lgfortran


Notice that the final link is done with g++ to get the C++ runtime libraries and -lgfortran is added to get the Fortran runtime libraries. In general, when mixing languages in LTO mode, you should use the same link command used when mixing languages in a regular (non-LTO) compilation. This means that if your build process was mixing languages before, all you need to add is -flto to all the compile and link commands.

If LTO encounters objects with C linkage declared with incompatible types in separate translation units to be linked together (undefined behavior according to ISO C99 6.2.7), a non-fatal diagnostic may be issued. The behavior is still undefined at runtime.

If object files containing GIMPLE bytecode are stored in a library archive, say libfoo.a, it is possible to extract and use them in an LTO link if you are using a linker with linker plugin support. To enable this feature, use the flag -fuse-linker-plugin at link-time:

gcc -o myprog -O2 -flto -fuse-linker-plugin a.o b.o -lfoo


With the linker plugin enabled, the linker will extract the needed GIMPLE files from libfoo.a and pass them on to the running GCC to make them part of the aggregated GIMPLE image to be optimized.

If you are not using a linker with linker plugin support and/or do not enable linker plugin then the objects inside libfoo.a will be extracted and linked as usual, but they will not participate in the LTO optimization process.

Link time optimizations do not require the presence of the whole program to operate. If the program does not require any symbols to be exported, it is possible to combine -flto and with -fwhole-program to allow the interprocedural optimizers to use more aggressive assumptions which may lead to improved optimization opportunities. Use of -fwhole-program is not needed when linker plugin is active (see -fuse-linker-plugin).

Regarding portability: the current implementation of LTO makes no attempt at generating bytecode that can be ported between different types of hosts. The bytecode files are versioned and there is a strict version check, so bytecode files generated in one version of GCC will not work with an older/newer version of GCC.

Link time optimization does not play well with generating debugging information. Combining -flto with -g is currently experimental and expected to produce wrong results.

If you specify the optional n, the optimization and code generation done at link time is executed in parallel using n parallel jobs by utilizing an installed make program. The environment variable MAKE may be used to override the program used. The default value for n is 1.

You can also specify -flto=jobserver to use GNU make's job server mode to determine the number of parallel jobs. This is useful when the Makefile calling GCC is already executing in parallel. The parent Makefile will need a `+' prepended to the command recipe for this to work. This will likely only work if MAKE is GNU make.

This option is disabled by default.
-flto-partition=alg
Specify the partitioning algorithm used by the link time optimizer. The value is either 1to1 to specify a partitioning mirroring the original source files or balanced to specify partitioning into equally sized chunks (whenever possible). Specifying none as an algorithm disables partitioning and streaming completely. The default value is balanced.
-flto-compression-level=n
This option specifies the level of compression used for intermediate language written to LTO object files, and is only meaningful in conjunction with LTO mode (-flto). Valid values are 0 (no compression) to 9 (maximum compression). Values outside this range are clamped to either 0 or 9. If the option is not given, a default balanced compression setting is used.
-flto-report
Prints a report with internal details on the workings of the link-time optimizer. The contents of this report vary from version to version, it is meant to be useful to GCC developers when processing object files in LTO mode (via -flto).

Disabled by default.
-fuse-linker-plugin
Enables the use of a linker plugin during link time optimization. This option relies on plugin support in the linker, which is available in gold or in GNU ld 2.21 or newer.

This option enables the extraction of object files with GIMPLE bytecode out of library archives. This improves the quality of optimization by exposing more code to the link time optimizer. This information specifies what symbols can be accessed externally (by non-LTO object or during dynamic linking). Resulting code quality improvements on binaries (and shared libraries that use hidden visibility) are similar to -fwhole-program. See -flto for a description of the effect of this flag and how to use it.

Enabled by default when LTO support in GCC is enabled and GCC was compiled with a linker supporting plugins (GNU ld 2.21 or newer or gold).
Regards

I forget...
Observe Ipo compressed absent
Well when updated ICC better future added an option "-Zippo" for result build the fire
of God
, and offer an equivalence have already Gnu compiler(-flto-compression-level=n)


avayasheik
Beginner
234 Views

H Tim;
Thanks for replying. We have redhat 5.2 so I cannot try the -lto option.
This is an evaluation version I am running - can I still get help from the premier.intel.com account?
avayasheik
Beginner
234 Views

Hi Om;
We need the ipo (that's one of the reasons we're contemplating Intel)
avayasheik
Beginner
234 Views

HI Bustaf;
Thanks for the lenghty reply. I tried the -fwhole-program option - but it fails to compile many other modules as well.
Dale_S_Intel
Employee
234 Views

Does it build properly if you disable auto-cpu dispatch (i.e. -ax)?
Dale
avayasheik
Beginner
234 Views

Hi Dale ;
I tried that - no luck - -ax is not the culprit.
I removed -ipo, -axSE4.2, and -msse2 and it builds sucessfully.

I then reenabled only -ipo and it fails.

-sheik
Dale_S_Intel
Employee
234 Views

There's a few other things to try, but none of them easy. I don't suppose it's feasible to submit the whole thing?
One option would be to try to figure out exactly which .o is causing the problem. You could do this by turning off -ipo for individual files until you find one file that makes a difference, but even then we're probably back to getting a test case of some sort in order to fix it.
Is this at all related to the problem you posted about last month, with the inline asm and the .single_core and .multi_core?
Dale
avayasheik
Beginner
234 Views

I'll do my best to turn off ipo individually - will take some time.
..and no, this is not related to the other issue. We have removed the asm code since.
-thanks
avayasheik
Beginner
234 Views

I was able to find the offending file. Not only that I was able to find the offending line as well.
The following code causes the ipo link error.

static char *Names[NumTypes] = {
NULL,
"string1",
"string2",
"string3",
"string4",
"string5",
"string6"
};

.
.
.

//if ( Names != NULL ) {
if(Names ! = "" ){

do something...
}

The above commented outline causes the error. If I replace with the line following,it links fine.
Withoutipo specified, thecommented outcase alsolinks fine.

Any thoughts?

Also, when compiling with -ipo, do I need to specify -ipo on the link line as well?
Thanks a bunch for all the help.

-sheik
JenniferJ
Moderator
234 Views

It is not so simple. I created a testcase with this code snippets, it does not show the ipo internal error.

It is great that you are able to identify the code and work-around the problem, but we still need a testcase in order to fix the bug in the compiler. It will likely take a lot longer time to create a testcase that will show the problem.

Please report it tothe Premier Support instead.

thanks,
Jennifer

Reply