Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Beginner
132 Views

Intel Compiler fails during final link (but g++ works)

I was trying to build node.js with intel compiler. All files compile okay, but the final linking stage fails. Strangely, if I use g++ for final link, it still work. The error message isn't quite helpful either. I have tried changing optimization from O3 to O2 - but no difference

-------------

": internal error: ** The compiler has encountered an unexpected problem. ** Segmentation violation signal raised. ** Access violation or stack overflow. Please contact Intel Support for assistance. icpc: error #10014: problem during multi-file optimization compilation (code 4)

-------

Appreciate any pointers ? Thanks!

 

0 Kudos
30 Replies
Highlighted
Moderator
119 Views

 

Can you provide us your build log to take look?

Thanks,

Viet

0 Kudos
Highlighted
Beginner
119 Views

I have the final linker log attached. Is there a way to get extended log ? On a different topic, are there any flags I need to use to make ICC compatible with GCC. I have great difficulty compiling projects that just work with GCC.

Edit: I also added all.log which is the log of entire build.

0 Kudos
Highlighted
Moderator
119 Views

What GCC version are you using? 

can you try to replace icpc manually with g++, to see if it link successfully? Dont rebuild your project, just try to link with g++

 

icpc (replace icpc with g++) -pthread -rdynamic -m64 -Wl,--whole-archive,/home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/deps/openssl/libopenssl.a -Wl,--no-whole-archive -Wl,-z,noexecstack -Wl,--whole-archive /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/deps/v8/src/libv8_base.a -Wl,--no-whole-archive -pthread  -o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/node -Wl,--start-group /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/async-wrap.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/cares_wrap.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/connection_wrap.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/connect_wrap.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/env.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/fs_event_wrap.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/handle_wrap.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/js_stream.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/module_wrap.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_api.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_buffer.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_config.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_constants.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_contextify.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_debug_options.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_file.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_http2.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_http_parser.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_main.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_os.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_platform.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_perf.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_serdes.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_url.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_util.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_v8.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_stat_watcher.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_watchdog.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_zlib.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_i18n.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/pipe_wrap.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/process_wrap.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/signal_wrap.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/spawn_sync.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/string_bytes.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/string_search.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/stream_base.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/stream_wrap.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/tcp_wrap.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/timer_wrap.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/tracing/agent.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/tracing/node_trace_buffer.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/tracing/node_trace_writer.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/tracing/trace_event.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/tty_wrap.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/udp_wrap.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/util.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/uv.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/gen/node_javascript.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/inspector_agent.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/inspector_io.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/inspector_js_api.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/inspector_socket.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/inspector_socket_server.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_crypto.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_crypto_bio.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/node_crypto_clienthello.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/tls_wrap.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/node/src/backtrace_posix.o /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/deps/nghttp2/libnghttp2.a /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/deps/v8/src/libv8_libplatform.a /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/tools/icu/libicui18n.a /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/deps/openssl/libopenssl.a /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/deps/zlib/libzlib.a /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/deps/http_parser/libhttp_parser.a /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/deps/cares/libcares.a /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/deps/uv/libuv.a /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/deps/v8/src/libv8_base.a /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/deps/v8/src/libv8_libbase.a /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/deps/v8/src/libv8_libsampler.a /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/tools/icu/libicuucx.a /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/tools/icu/libicudata.a /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/tools/icu/libicustubdata.a /home/ubuntu/cloudbin/node-v9.1.0.icc/out/Release/obj.target/deps/v8/src/libv8_snapshot.a -ldl -lrt -lm -Wl,--end-group

 

 

Thanks,

Viet 

0 Kudos
Highlighted
Beginner
119 Views

Viet - Yes. replacing icpc with g++ does work (ofcourse, I need to link libirc and libimf). g++ version is 5.4.0

There are other issues with my other projects too. GCC works just fine - but there is some issue or other. For example, there is one involving builtins available in gcc, but not icc.

error: argument is incompatible with formal parameter return __builtin_mul_overflow(lhs, rhs, product);

Are there any best practises to make icc/icpc become gcc compatible ? 

0 Kudos
Highlighted
Moderator
119 Views

Is it possible to provide all of your objects and dependency libs so that we can reproduce the issue at our end? 

_builtin_mul_overflow(...) is available in ICC as well. Seems like you have miss-matched between argument and parameter.

Thanks,

Viet 

 

0 Kudos
Highlighted
Beginner
119 Views

Viet Yes - I can provide. Is there an email I can send those (it could be huge) ?

0 Kudos
Highlighted
119 Views

>>problem during multi-file optimization compilation (code 4)

icc with any optimization level enables by default multi-file interprocedural optimizations. Try disabling multi-file IPO (-no-ipo). If that works then try adding back intra-file interprocedural optimizations (-ip).

If that works then you can potentially compose your application into two parts, those that can have multi-file interprocedural optimizations, and those that cannot (you may need to place those that cannot into an .so or lib file).

>>Seems like you have miss-matched between argument and parameter.

Follow the advice given by Viet and determine the cause for the error.

Jim Dempsey

0 Kudos
Highlighted
Moderator
119 Views

can you tar it up and attach here?

Hi Jim,

I don't think objects were built with -ipo as mentioned that it was able to link with g++. ipo objects are required to use icc to link.

Thanks,

Viet

 

0 Kudos
Highlighted
Beginner
119 Views

Jim - Thanks. Turning off ipo did help in getting a successful build. But adding back "-ip" gave the same error. I am actually okay to not use "-ip" (assuming the individual files are compiled with best flags - so final linking optimization may only offer incremental performance..I am guessing here though)

0 Kudos
Highlighted
Beginner
119 Views

Viet - Please see https://s3.amazonaws.com/icc_projects/icc_error.tar.gz for the entire tar bundle. It contains everything you would need. Thanks for taking a look.

0 Kudos
Highlighted
Moderator
119 Views

 

Not sure what happened, but I couldn't be able to link

vahoang@orcsle147:/tmp$ sh vah_icc.sh
/tmp/vah/icc_errors/issue1_ipoissue/node-v9.1.0.icc/out/Release/obj.target/deps/nghttp2/libnghttp2.a: could not read symbols: Malformed archive

 

 

0 Kudos
Highlighted
Beginner
119 Views

Viet - Could you try rebuilding again ?

make clean

 ./configure

make CC=icc CXX=icpc -j35 [35 threads or whatever #of cores you have] 

This will go through until the final link stage. The other easier option is for me to give access to my system so you can poke around. Let me know if it's preferable

0 Kudos
Highlighted
Moderator
119 Views

 

For issue1: I saw the error if I rebuild it. I'll take me sometime to investigate though. I will let you know 

For  issue2, I was able to compile. I didn't see it complained about builtin_mul_overflow(...)

vahoang@orcsle147:/tmp/vah/icc_errors/issue2_nobuildin$ sh vah.sh
vahoang@orcsle147:/tmp/vah/icc_errors/issue2_nobuildin$ ls -lt explain_cmd.o
-rw-r--r-- 1 vahoang ome 4184168 Nov 13 19:27 explain_cmd.o
vahoang@orcsle147:/tmp/vah/icc_errors/issue2_nobuildin$ cat vah.sh
icpc -o explain_cmd.o -c -std=c++14 -fno-omit-frame-pointer -fno-strict-aliasing -ggdb -pthread -fstack-protector-strong -fno-builtin -memcmp -fPIE -DPCRE_STATIC -DNDEBUG -D_FORTIFY_SOURCE=2 -DBOOST_SYSTEM_NO_DEPRECATED -DBOOST_MATH_NO_LONG_DOUBLE_MATH_FUNCTIONS -Isrc/third_party/s2 -Isrc/third_party/pcre-8.41 -Isrc/third_party/boost-1.60.0 -Ibuild/opt -Isrc src/mongo/db/commands/explain_cmd.cpp -w -diag-disable:remark
vahoang@orcsle147:/tmp/vah/icc_errors/issue2_nobuildin$

 

Thanks,

Viet

0 Kudos
Highlighted
Beginner
119 Views

That's great to now. My ICC is as below. Are we using the same version ?

$ /opt/intel/compilers_and_libraries_2018.0.128/linux/bin/intel64/icc --version icc 

(ICC) 18.0.0 20170811 Copyright (C) 1985-2017 Intel Corporation. All rights reserved.

I still get an error as attached.

 

 

0 Kudos
Highlighted
Moderator
119 Views

Yes, I used the same compiler version because we only have one ICC 18.0 released version.

$ icc -V
Intel(R) C Intel(R) 64 Compiler for applications running on Intel(R) 64, Version 18.0.0.128 Build 20170811
Copyright (C) 1985-2017 Intel Corporation.  All rights reserved.

 

Not sure why you got the error, I tried on a few different machines, it compiled successfully. I got some warnings if I removed -w.

$ sh vah.sh
icpc: command line warning #10159: invalid argument for option '-m'
In file included from /nfs/pdx/disks/cts2/tools/gcc/gcc-5.3.0/bin/../include/c++/5.3.0/x86_64-unknown-linux-gnu/bits/os_defines.h(39),
                 from /nfs/pdx/disks/cts2/tools/gcc/gcc-5.3.0/bin/../include/c++/5.3.0/x86_64-unknown-linux-gnu/bits/c++config.h(482),
                 from /nfs/pdx/disks/cts2/tools/gcc/gcc-5.3.0/bin/../include/c++/5.3.0/new(39),
                 from src/third_party/boost-1.60.0/boost/optional/optional.hpp(20),
                 from src/third_party/boost-1.60.0/boost/optional.hpp(15),
                 from src/mongo/db/commands.h(31),
                 from src/mongo/db/commands/explain_cmd.cpp(31):
/usr/include/features.h(330): warning #1224: #warning directive: _FORTIFY_SOURCE requires compiling with optimization (-O)
  #  warning _FORTIFY_SOURCE requires compiling with optimization (-O)
     ^

In file included from src/mongo/db/dbmessage.h(37),
                 from src/mongo/db/query/canonical_query.h(33),
                 from src/mongo/db/query/explain.h(33),
                 from src/mongo/db/commands.h(43),
                 from src/mongo/db/commands/explain_cmd.cpp(31):
src/mongo/db/server_options.h(192): warning #858: type qualifier on return type is meaningless
          const bool isVersionInitialized() const {
          ^

In file included from src/mongo/db/dbmessage.h(37),
                 from src/mongo/db/query/canonical_query.h(33),
                 from src/mongo/db/query/explain.h(33),
                 from src/mongo/db/commands.h(43),
                 from src/mongo/db/commands/explain_cmd.cpp(31):
src/mongo/db/server_options.h(200): warning #858: type qualifier on return type is meaningless
          const Version getVersion() const {
          ^

In file included from src/mongo/db/dbmessage.h(37),
                 from src/mongo/db/query/canonical_query.h(33),
                 from src/mongo/db/query/explain.h(33),
                 from src/mongo/db/commands.h(43),
                 from src/mongo/db/commands/explain_cmd.cpp(31):
src/mongo/db/server_options.h(214): warning #858: type qualifier on return type is meaningless
          const bool isSchemaVersion36() {
          ^

src/mongo/bson/util/builder.h(292): warning #2196: routine is both "inline" and "noinline"
              grow_reallocate(minSize);
                             ^
          detected during instantiation of "char *mongo::_BufBuilder<BufferAllocator>::grow(int) [with BufferAllocator=mongo::SharedBufferAllocator]" at line 138 of "src/mongo/bson/bsonobjbuilder.h"

 

0 Kudos
Highlighted
Beginner
119 Views

Something is wrong. So I added -M option to the command and it spits out the dependencies as below. Anything getting messed up in the includes may be ? I am clueless as well.

 

------

explain_cmd.o: src/mongo/db/commands/explain_cmd.cpp /usr/include/stdc-predef.h \
 src/mongo/platform/basic.h src/mongo/db/commands.h \
 src/third_party/boost-1.60.0/boost/optional.hpp \
 src/third_party/boost-1.60.0/boost/optional/optional.hpp /usr/include/c++/5/new \
 /usr/include/x86_64-linux-gnu/c++/5/bits/c++config.h \
 /usr/include/x86_64-linux-gnu/c++/5/bits/os_defines.h /usr/include/features.h \

[--Edited out rest to reduce clutter----]

0 Kudos
Highlighted
Beginner
119 Views

Actually adding -H gives the include path as below

. /usr/include/stdc-predef.h
. src/mongo/platform/basic.h
. src/mongo/db/commands.h
.. src/third_party/boost-1.60.0/boost/optional.hpp
... src/third_party/boost-1.60.0/boost/optional/optional.hpp
 

[--Edited out rest to reduce clutter----]

0 Kudos
Highlighted
119 Views

I notice that you are using boost. Boost templates predate many of the newer std:: templates, in particular std::enable_if, and those often use within enable_if to qualify expansion. IOW, if you omit std:: or boost::, you may end up with the wrong (unintended) template.

Due to the error seemingly related to return type, and/or argument type, you may have template expansion issues.

I suggest you examine all occurrences of enable_if to assure it, and enclosed templates, are properly scoped.

Jim Dempsey

0 Kudos
Highlighted
Beginner
119 Views

Jim - Strangely, the build does work for Viet without any changes. We are using the same icc compiler version too.

0 Kudos