Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
141 Views

Regression of Intel C++ Compiler v19.1 : Unresolved symbol

Using Ubuntu 19.10 or 18.04 LTS, when using the latest version of the Intel C++ Compiler issued in Parallel Studio XE 2020 (version 19.1.0.166 / 20191121), attempts to build UCX with the -ipo flag result in failure when compiling & linking sockaddr (sa); this appears to be an issue with Intel's Compiler rather than UCX because the 2018 and 2019 editions of Parallel Studio XE (versions 18.0.6.286 / 20190605 and 19.0.5.281 / 20190815, respectively) are able to compile and link it just fine. Using GCC and G++ with LTO enabled is also able to compile and link it without issue. Also attached are the entire config and build logs.

These are the symbols that are not found. Attempting to explicitly link with -lstdc++, using different C++ and GNU++ standards, etc., fail to fix the problem:

ipo: error #11021: unresolved _ZNSt14_Function_base12_M_max_alignE
        Referenced in /tmp/ipo_icpcYJnRpy.o
ipo: error #11021: unresolved _ZNSt14_Function_base11_M_max_sizeE
        Referenced in /tmp/ipo_icpcYJnRpy.o

These are the demangled symbols:

std::_Function_base::_M_max_align
std::_Function_base::_M_max_size

An extremely easily reproduceible example that will fail to build is given below:

1. git clone --recurse-submodules https://github.com/openucx/ucx.git && cd ucx
2. ./autogen.sh
3. ./configure CC=icc CXX=icpc LD=xild AR=xiar CFLAGS="-ipo"
4. make -j$(nproc)

This is the open bug report on the UCX project (where it was stated to likely be a regression on Intel's part), detailing the same information covered here:

https://github.com/openucx/ucx/issues/4624

0 Kudos
12 Replies
Highlighted
Moderator
141 Views

Is there another way to reproduce it? I couldn't connect from my end

orcsle138:/tmp/vah1$ git clone --recurse-submodules https://github.com/openucx/ucx.git
Cloning into 'ucx'...
fatal: unable to access 'https://github.com/openucx/ucx.git/': Failed to connect to github.com port 443: Connection timed out
orcsle138:/tmp/vah1$
 

0 Kudos
Highlighted
141 Views

Viet Hoang (Intel) wrote:

fatal: unable to access 'https://github.com/openucx/ucx.git/': Failed to connect to github.com port 443: Connection timed out
orcsle138:/tmp/vah1$

Hm. Not sure why github wouldn't clone a repo for you, but it should work with any of the source code distributions like this one specifically https://github.com/openucx/ucx/releases/download/v1.7.0/ucx-1.7.0.tar.gz

And if your environment has a JDK, you may want to add the "--without-java" flag to the configure script. Also, as of yesterday, the issue was closed on github as they were able to reproduce the issue & determined it was an issue with the 2020 edition of the compiler.

0 Kudos
Highlighted
Moderator
141 Views

Thanks for the info.

0 Kudos
Highlighted
141 Views

Viet Hoang (Intel) wrote:

Thanks for the info.

Ended up editing the info a bit as the link I provided did not reproduce it initially; not sure which edit you saw, so just letting you know it may have changed.  At the moment, I am having trouble reproducing it with anything but the git clone, but perhaps it will still do it for you without it.

0 Kudos
Highlighted
141 Views

Sorry for the third post now and all of the edits :( ; was trying to respond to you quickly and just made it worse!

I have confirmed you can reproduce the error by the following

1. Download this https://github.com/openucx/ucx/archive/master.zip

2. unzip it

3. cd to the directory, do the commands "git init" and "git submodule" before doing ./autogen.sh

4. run this : ./configure --without-java CC=icc CXX=icpc LD=xild AR=xiar CFLAGS="-ipo0"

5. You don't need to build the entire program. after configuring, cd to test/apps/sockaddr from the ucx source dir. Run make from here. You should get the error.

0 Kudos
Highlighted
Moderator
141 Views

This is not a regression. New compiler version emits a warning, but you have -Werror to force warnings to be reported as error. so you need to add  -wd11021 to makefile, or remove -Werror

Also note that -g without any -O flags will be default to -O0.

/tmp/vah/ucx-master/test/apps/sockaddr$ make clean; make
rm -rf .libs _libs
 rm -f sa
rm -f *.o
rm -f *.lo
  CXX      sa-sa_base.o
  CXX      sa-sa_main.o
  CXX      sa-sa_tcp.o
  CXX      sa-sa_util.o
  CXXLD    sa
ipo: error #11021: unresolved _ZNSt14_Function_base12_M_max_alignE
        Referenced in /tmp/ipo_icpcxPdPOt.o
ipo: error #11021: unresolved _ZNSt14_Function_base11_M_max_sizeE
        Referenced in /tmp/ipo_icpcxPdPOt.o
icpc: error #10014: problem during multi-file optimization compilation (code 1)
make: *** [sa] Error 1
:/tmp/vah/ucx-master/test/apps/sockaddr$

:/tmp/vah/ucx-master/test/apps/sockaddr$ diff Makefile Makefile.bak
437c437
<       -std=c++11 -g -Wall -Werror -wd11021 -O3
---
>       -std=c++11 -g -Wall -Werror

:/tmp/vah/ucx-master/test/apps/sockaddr$ make clean; make
rm -rf .libs _libs
 rm -f sa
rm -f *.o
rm -f *.lo
  CXX      sa-sa_base.o
  CXX      sa-sa_main.o
  CXX      sa-sa_tcp.o
  CXX      sa-sa_util.o
  CXXLD    sa

:/tmp/vah/ucx-master/test/apps/sockaddr$

 

0 Kudos
Highlighted
141 Views

Yes, you can make it compile by not using -Werror or forcing it to ignore that particular warning.  Can you please provide more information on why this is not a regression? Unresolved symbols were reported in previous versions of the compiler, too. As far as I can tell, error #11021 is the error issued for undefined reference/unresolved symbols, and it is not a new error message in the 2020 compiler. Why would the compiler not have been able to detect this particular symbol in previous editions? And when using previous versions, this error is not emitted even though -Werror is used. Even forcing static links, using --whole-archive or --start-group/--end-group does not fix it.

I also see that such errors from ipo are not unprecedented: https://software.intel.com/en-us/articles/ipo-warning-11021-unresolved-symbols-referenced-a-dynamic-library

But if this is indeed not a 

Viet Hoang (Intel) wrote:

Also note that -g without any -O flags will be default to -O0.

Also, at least for me, when I issue the "-ipo" flag, the "-O3" is automatically appended to the generated Makefile(s); so, after running configure, test/apps/sockaddr/Makefile contains -O3.

 

If you can confirm this is truly not a regression and provide more details on why this is not a regression when GCC's LTO and previous versions of Intel's compiler do not have an issue with this symbol being unresolved, that would be great so that I can re-open the bug with ucx.

0 Kudos
Highlighted
Moderator
141 Views

Please provide us a simple test case, which shows the error without promoting a warning to an error. 

0 Kudos
Highlighted
141 Views

Viet Hoang (Intel) wrote:

Please provide us a simple test case, which shows the error without promoting a warning to an error. 

If you generate and view the IPO report at a level 4 or 5 , whole-program optimization will have failed. Using IPO reporting on previous versions of the compiler will show that WPO was successful. I still don't understand why you are saying that this is simply the new type of error that can be reported in the 2020 edition. This error field is not new, and I find it odd to suggest that an old version of the compiler that was incapable of such a basic task as reporting unresoved symbols; furthermore, this is not even listed as an improvement/errata for the 2020 edition. Your suggested fix of removing the flags and adding a flag to ignore that error code only allows the compilation to succeed. So I suppose my opening post was wrong to suggest that the Intel compiler is to blame for the actual FAILED compilation, as that is not the case. That doesn't change the fact that this is warning exists and that IPO is not performing as well as it could and should. 

 

I provide two examples below. However, I want to make it emphatically clear that I have found that using ANY C++ CODE that contains std::function and a call to "#include <functional>" will result in this symbol being unresolved when using IPO, only in the latest edition of the compiler. (So, e.g., the example usage http://www.cplusplus.com/reference/functional/function/function/ will fail with ipo and the steps listed below to receive the warning).

In two links below here are two unrelated (unrelated to eachother or ucx) c++ files, other than that they both invoke the functional header, which is what ICPC ipo is having problems with. With either of them, if you check in previous versions, this error does not happen. To be notified of these unresolved symbols, you must compile with either "-g" and/or with "-qopt-report=4" or 5. As mentioned before, in the 2020 version, you will receive a report stating "FALSE" for whole program optimization. (Excerpts from these reports are at the end of this post).

Simple Example: https://controlc.com/e5388f22

Long Example: https://controlc.com/b60eaa31

To reproduce on either file:

icpc -ipo -g -c -o foo.o foo.cc
OR
icpc -ipo -qopt-report=5 -c -o foo.o foo.cc

Followed by this to receive the error:
icpc -o foo foo.o

Here are the above mentioned excerpts from ipo reports:

Intel(R) C++ Intel(R) 64 Compiler for applications running on Intel(R) 64, Version 18.0.5.274 Build 20180823

Compiler options: -std=c++11 -g -Wall -Werror -ipo -qopt-report-phase=ipo -qopt-report=5 -no-inline-max-total-size -no-inline-max-per-compile -no-inline-max-size -no-inline-max-per-routine -qoverride-limits -o sa -lpthread -lrt -lbfd -ldl

    Report from: Interprocedural optimizations [ipo]

  WHOLE PROGRAM (SAFE) [EITHER METHOD]: true
  WHOLE PROGRAM (SEEN) [TABLE METHOD]: false
  WHOLE PROGRAM (READ) [OBJECT READER METHOD]: true

As compared to the latest release of the compiler:

Intel(R) C++ Intel(R) 64 Compiler for applications running on Intel(R) 64, Version 19.1.0.166 Build 20191121

Compiler options: -std=c++11 -g -Wall -Werror -ipo -qopt-report-phase=ipo -qopt-report=5 -no-inline-max-total-size -no-inline-max-per-compile -no-inline-max-size -no-inline-max-per-routine -qoverride-limits -o sa -lpthread -lrt -lbfd -ldl

    Report from: Interprocedural optimizations [ipo]

  WHOLE PROGRAM (SAFE) [EITHER METHOD]: false
  WHOLE PROGRAM (SEEN) [TABLE METHOD]: false
  WHOLE PROGRAM (READ) [OBJECT READER METHOD]: false

 

0 Kudos
Highlighted
Moderator
141 Views

I still don't understand why you are saying that this is simply the new type of error that can be reported in the 2020 edition.

Again, it's a warning, not an error. Does your executable behave differently with this warning?

0 Kudos
Highlighted
141 Views

Sorry, I do not mean to keep referring to it as an error. I know it is a warning.

I'm really not sure how report on this much further. I am not a programmer, per se, by trade, and so going into this level of detail where I have to dig into libstdc++, gcc, their headers, etc, becomes quickly overwhelming for me. I really don't have a better example than this to just receive the warning. As said, previous versions report whole-program optimization and do not fail to resolve the symbol. This one does. I (think) these unresolved symbols seem to be related to the maximum number of elements (_M_max_size) and the maximum alignment byte size for functors. These vary by system, compiler, etc, I believe, but I would think it would cause issues should the Linux version /w GCC, and without boost, ever attempts to make a functor with an alignment greater than the default e.g. 8 bytes, or more than 16 elements. In the following example, the actual compiled test seems to have no issue actually resolving _M_max_*, regardless of what the compiler & linker think. The reported alignments also change with -Zp.

Either way, even if the compilation is successful/without error, I don't see how a newer version of the compiler would not be considered regressed when it fails to resolve symbols previous versions could and to optimize as well as previous versions. If performance regressions are not of concern to you all, so be it.

Small test programs using std::function seem to report the same values, on my singular Ubuntu system at least, but I am not going to benchmark across different systems with a theoretically endless set of different programs using functors to see if one of them results in undefined/incorrect behavior, nor to see just how much the performance is reduced by the fewer optimziations applied.

#include <iostream>
#include <functional>

int main(){
    std::cout << "sizeof and __alignof(_Nocopy_types) must be <= _M_max_*" << std::endl;

    std::cout << "sizeof: " << sizeof(std::_Nocopy_types) << std::endl;
    std::cout << "_M_max_size: " << std::_Function_base::_M_max_size << std::endl;

    std::cout << "__alignof__: " << __alignof__(std::_Nocopy_types) << std::endl;
    std::cout << "_M_max_size: " << std::_Function_base::_M_max_align << std::endl;
}

 

0 Kudos
Highlighted
141 Views

Well since Intel is blissfully indignant towards to their shortcoming in the C++11 standard, hopefully this issue won't be present in the next version of the compiler, but for anyone who comes across this issue from Googling, etc., the following will fix the warning & allow for full IPO to take place (even though one should ideally never try and redefine these things from system headers). You will need to use the -ansi flag or -std=c++98, and manually define needed segments not included by default from std_function when using c++98.

class _Undefined_class;
union _Nocopy_types
  {
    void*       _M_object;
    const void* _M_const_object;
    void (*_M_function_pointer)();
    void (_Undefined_class::*_M_member_pointer)();
  };
static const std::size_t _M_max_size = sizeof(_Nocopy_types);
static const std::size_t _M_max_align = __alignof__(_Nocopy_types);

Example:

#include <functional>
class _Undefined_class;
union _Nocopy_types
  {
    void*       _M_object;
    const void* _M_const_object;
    void (*_M_function_pointer)();
    void (_Undefined_class::*_M_member_pointer)();
  };
static const std::size_t _M_max_size = sizeof(_Nocopy_types);
static const std::size_t _M_max_align = __alignof__(_Nocopy_types);
#include <iostream>

int main(){
    std::cout << "sizeof: " << sizeof(_Nocopy_types) << std::endl;
    std::cout << "_M_max_size: " << _M_max_size << std::endl;

    std::cout << "__alignof__: " << __alignof__(_Nocopy_types) << std::endl;
    std::cout << "_M_max_align: " << _M_max_align << std::endl;
}

It appears to work correctly as adding directives such as -Zp4 adjust the output appropriate, and with -Zp16, it still appropriately restricts it to 8, but use at your own risk. The symbols also appear not to be defined in the final binary, which I believe they are not supposed to be, so this fix doesn't seem to alter that.

In the preprocessed file for my suggested fix + -ansi flag, the majority of null_pointer typedefs and nullptr are altogether absent, all references to polymorphism are missing. Comparing the ipo reports, in just my extremely small example, an additional 3.7% of the program was inlined.

 

0 Kudos