Intel® Integrated Performance Primitives
Deliberate problems developing high-performance vision, signal, security, and storage applications.
6704 Discussions

Illegal instruction in libiomp5 when upgrading from ipp 7.0.7

Francesco_Lamonica
974 Views
Hi all,
i am running an IPP based application on a GEODE cpu running a linux (debian etch) 32bit distribution.
With ipp up to 7.0.5 all is fine. I recently upgraded to IPP 7.0.7 and i get an illegal instruction at startup
the backtrace i get is the following:
(gdb) r
Starting program: /opt/agent/testingprog
[Thread debugging using libthread_db enabled]
[New Thread -1223795808 (LWP 12265)]
Program received signal SIGILL, Illegal instruction.
[Switching to Thread -1223795808 (LWP 12265)]
rml::internal::internalPoolMalloc (mPool=0xb76f2540, size=11) at ../../src/tbbmalloc/frontend.cpp:1917
1917 ../../src/tbbmalloc/frontend.cpp: No such file or directory.
in ../../src/tbbmalloc/frontend.cpp
(gdb) where
#0 rml::internal::internalPoolMalloc (mPool=0xb76f2540, size=11) at ../../src/tbbmalloc/frontend.cpp:1917
#1 0xb76d0b6e in scalable_malloc (size=11) at ../../src/tbbmalloc/frontend.cpp:2194
#2 0xb7682ad2 in __kmp_env_get () from /opt/agent/lib/libiomp5.so
#3 0xb768804c in __kmp_i18n_catopen () from /opt/agent/lib/libiomp5.so
#4 0xb7688950 in __kmp_i18n_catgets () from /opt/agent/lib/libiomp5.so
#5 0xb7688852 in __kmp_msg_format () from /opt/agent/lib/libiomp5.so
#6 0xb7690858 in __kmp_rml_report_error_handler () from /opt/agent/lib/libiomp5.so
#7 0xb76b1c4a in __kmp::dynamic_link () from /opt/agent/lib/libiomp5.so
#8 0xb769ac45 in __kmp_do_serial_initialize () from /opt/agent/lib/libiomp5.so
#9 0xb769ab66 in __kmp_middle_initialize () from /opt/agent/lib/libiomp5.so
#10 0xb7684435 in omp_get_num_procs () from /opt/agent/lib/libiomp5.so
#11 0xb762b9d6 in _init () from /opt/agent/lib/libippcore.so.7.0
#12 0xbf826278 in ?? ()
#13 0xb764f4c8 in _init () from /opt/agent/lib/libiomp5.so
#14 0xb7f1d921 in _dl_rtld_di_serinfo () from /lib/ld-linux.so.2
#15 0xb7f127ef in ?? () from /lib/ld-linux.so.2
0 Kudos
18 Replies
Gennady_F_Intel
Moderator
974 Views
this is an unknown issue - both of these versions contain the same versions of threading libraries( libiomp ) if I remember exactly. you can check it by himself. what you were changing in your application after IPP's binaries were updated? Did you change linking line?
0 Kudos
Francesco_Lamonica
974 Views
Hi Gennady,
executing a diff between the two libiomp5.so files returns a difference.
The scenario is the following
testapp links against testlib
testlib uses libspeech.a (from ipp-samples) that uses libipps and libippsc functions.
(libspeech was obviously recompiled from 7.0.5 and 7.0.7)
the code (and Makefile) of the testlibrary has not changed
the code (and Makefile) of the testapp has not changed.
besides from what i gather from stack trace this seems to happen before the testlib is being loaded...
let me know if you need further info
0 Kudos
Francesco_Lamonica
974 Views
Any news about this topic?
0 Kudos
Gennady_F_Intel
Moderator
974 Views

We cannot reproduce the issue on our side.can you give the reproducible test for us?

0 Kudos
Francesco_Lamonica
974 Views
Hi Gennady,
the actual code can be reduced to a simple
#include
int main() {
std::cout << "Hello world" << std::endl;
return 0;
}
this simple code linked against libiomp5, libippcore
with the line: g++ -Wall -g test.cpp -L -liomp5 -lippcore -o test
it was compiled with gcc 4.1.2 (debian etch on geode) and works with ipp 7.0.5, but gives an illegal instruction with ipp 7.0.7
etch:/# ldd ./test-ipp707
linux-gate.so.1 => (0xb7f31000)
libiomp5.so => /root/libiomp5.so (0xb7e73000)
libippcore.so.7.0 => /root/libippcore.so.7.0 (0xb7e5c000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7d73000)
libm.so.6 => /lib/tls/libm.so.6 (0xb7d4e000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7d43000)
libc.so.6 => /lib/tls/libc.so.6 (0xb7c10000)
libpthread.so.0 => /lib/tls/libpthread.so.0 (0xb7bfe000)
libdl.so.2 => /lib/tls/libdl.so.2 (0xb7bfa000)
/lib/ld-linux.so.2 (0xb7f32000)
etch:/# ./test-ipp707
Illegal instruction
0 Kudos
Ying_H_Intel
Employee
974 Views
Hi Francesco,

Seeing fromthe simpletest code, it seems nothing to do with the libiomp5 (openmp library)and ipp libraries. But anyway, could you pleaseshow the gdb resultwhen run the "test-ipp707"?

(in the test code,the"test" is"test-ipp707", right?). and please export theOpenMP version alsoas below.
[U105439illegal]$ export KMP_VERSION=1
[U105439illegal]$ gdb test
Thanks
Ying
0 Kudos
Francesco_Lamonica
974 Views
Hello Ying,
here is what you asked for :)
/tmp/ipp-illegal-instruction-test# gdb ./test
GNU gdb 6.4.90-debian
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i486-linux-gnu"...Using host libthread_db library "/lib/tls/libthread_db.so.1".
(gdb) r
Starting program: /rw/tmp/ipp-illegal-instruction-test/test
[Thread debugging using libthread_db enabled]
[New Thread -1211778208 (LWP 11595)]
Program received signal SIGILL, Illegal instruction.
[Switching to Thread -1211778208 (LWP 11595)]
rml::internal::internalPoolMalloc (mPool=0xb7f87540, size=11) at ../../src/tbbmalloc/frontend.cpp:1917
1917 ../../src/tbbmalloc/frontend.cpp: No such file or directory.
in ../../src/tbbmalloc/frontend.cpp
(gdb)
However i do not understand why you say it has nothing to do with iomp5 and ipp... the simple snippet does (almost) nothing and we get the illegal instruction only when is linked against libiomp5 and ipp. The stack trace shows how the calls from the loader are up to the libiomp malloc call (before any actual code is being run)
Or am i reading the stack trace wrong?
thanks
0 Kudos
Ying_H_Intel
Employee
974 Views
Hi Francesco,

Thansk for the test. i see. I assumed, if without libiomp5 and libippcore, the command: g++ -Wall -g test.cpp -o test should be work fine. sothe codehas nothing to do with libiomp5 and ipp.But the binary did.


Could you pleasetry the command :
1)g++ -Wall -g test.cpp -L -liomp5 -o test
2)g++ -Wall -g test.cpp -L -lippcore -liomp5-o test

andseeifthey can run?

We want to make sure if it is libiomp5 issue or ipp issue orthe combination's order?
and pleaseset environment export KMP_VERSION=1 before run them.

Best Regards,
Ying
0 Kudos
Pavel_B_Intel1
Employee
974 Views
Hello Francesco,

do you use standalone IPP package or Intel Composer 2011? did you update IPP verion only or Intel Composer too?

Pavel
0 Kudos
Francesco_Lamonica
974 Views
Hi Ying,
here is the report of the tests
== On the Host ==

vm-etch-dev32# cat ipp_test.cpp
#include

int main(int argc, char** argv)
{
std::cout << "testing ipp" << std::endl;
return 0;
}

vm-etch-dev32# g++ -Wall -g ipp_test.cpp -L./ -liomp5 -o ipp_test_1

vm-etch-dev32# g++ -Wall -g ipp_test.cpp -L./ -lippcore -liomp5 -o ipp_test_2

vm-etch-dev32# ls
ipp_test ipp_test_1 ipp_test_2 ipp_test.cpp libiomp5.so libippcore.so libippcore.so.7.0 test

== On the Target ==

etch:/tmp/ipp-illegal-instruction-test# ldd ipp_test_1
linux-gate.so.1 => (0xb7f8c000)
libiomp5.so => /tmp/ipp-illegal-instruction-test/libiomp5.so (0xb7ece000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7de5000)
libm.so.6 => /lib/tls/libm.so.6 (0xb7dc0000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7db5000)
libc.so.6 => /lib/tls/libc.so.6 (0xb7c83000)
libpthread.so.0 => /lib/tls/libpthread.so.0 (0xb7c70000)
libdl.so.2 => /lib/tls/libdl.so.2 (0xb7c6c000)
/lib/ld-linux.so.2 (0xb7f8d000)

etch:/tmp/ipp-illegal-instruction-test# ldd ipp_test_2
linux-gate.so.1 => (0xb7ef9000)
libippcore.so.7.0 => /tmp/ipp-illegal-instruction-test/libippcore.so.7.0 (0xb7ee0000)
libiomp5.so => /tmp/ipp-illegal-instruction-test/libiomp5.so (0xb7e24000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7d3b000)
libm.so.6 => /lib/tls/libm.so.6 (0xb7d16000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7d0b000)
libc.so.6 => /lib/tls/libc.so.6 (0xb7bd8000)
libdl.so.2 => /lib/tls/libdl.so.2 (0xb7bd4000)
libpthread.so.0 => /lib/tls/libpthread.so.0 (0xb7bc2000)
/lib/ld-linux.so.2 (0xb7efa000)

etch:/tmp/ipp-illegal-instruction-test# ./ipp_test_1
testing ipp
etch:/tmp/ipp-illegal-instruction-test# ./ipp_test_2
Illegal instruction
so it seems the problem lies in libippcore
0 Kudos
Francesco_Lamonica
974 Views
Hi Pavel,
i am using only IPP, the compiler has always been gcc 4.1.2
0 Kudos
Pavel_B_Intel1
Employee
974 Views
Francesco,

unfortunatelly I don't have appropriate HW to test your case. Could you help me and build 3 variant of application:
1) build your test application and link with libippcore.so only
2) build your test application and link with libiomp5.so only
3) extend your test application to add:

extern int omp_get_num_procs (void);
.....
main()
{
....
int p= omp_get_num_procs();
....
}

and link it with libiomp5.so only

Thanks, Pavel
0 Kudos
Francesco_Lamonica
974 Views
Hi Pavel,
as you can see from my post #10 case 2) was already tested and it printed the correct string
i tried now also case 1) and case 3) and the results are:
case 1: i get the illegal instruction
case 3: i could not test on the geode board since it did not compile
vm-etch-dev32# cat ipp_test_intel_get_procs.cpp
#include

extern int omp_get_num_procs (void);

int main(int argc, char** argv)
{
int p = omp_get_num_procs();
std::cout << "testing ipp: no of procs is" << p << std::endl;
return 0;
}

vm-etch-dev32# g++ -Wall -g ipp_test_intel_get_procs.cpp -L./ -liomp5 -o ipp_test_get_procs
/tmp/cckp3bTU.o: In function `main':
/root/sandbox/ipp-illegal-instruction-test/ipp_test_intel_get_procs.cpp:7: undefined reference to `omp_get_num_procs()'
collect2: ld returned 1 exit status

to sum it up:
vm-etch-dev32# cat ipp_test.cpp
#include

int main(int argc, char** argv)
{
std::cout << "testing ipp" << std::endl;
return 0;
}

vm-etch-dev32# g++ -Wall -g ipp_test.cpp -L./ -liomp5 -o ipp_test_iomp5_only
vm-etch-dev32# g++ -Wall -g ipp_test.cpp -L./ -lippcore -o ipp_test_ippcore_only
vm-etch-dev32# g++ -Wall -g ipp_test.cpp -L./ -lippcore -liomp5 -o ipp_test_ippcore_and_iomp5

etch:/tmp/ipp-illegal-instruction-test# ./ipp_test_iomp5_only
testing ipp
etch:/tmp/ipp-illegal-instruction-test# ./ipp_test_ippcore_only
Illegal instruction
etch:/tmp/ipp-illegal-instruction-test# ./ipp_test_ippcore_and_iomp5
Illegal instruction
the one with the omp_get_num_procs() did not compile
0 Kudos
Pavel_B_Intel1
Employee
974 Views
Hi Francesco,

sorry to disturb you the right source is:

#include

extern "C" {
extern int omp_get_num_procs (void);
}

int main(int argc, char** argv)
{
int p = omp_get_num_procs();
std::cout << "testing ipp: no of procs is" << p << std::endl;
return 0;
}

g++ test.cpp -o test -L /nfs/site/proj/ipp/build/ipp/ipp70gold-20120406-7-0-0205-105/comp_lib/linux32 -liomp5 -lpthread

one more question: why you don't link pthread library in your tests? libiomp5.so required pthread library:

g++ test.cpp -o test -L /nfs/site/proj/ipp/build/ipp/ipp70gold-20120406-7-0-0205-105/comp_lib/linux32 -liomp5

/nfs/site/proj/ipp/build/ipp/ipp70gold-20120406-7-0-0205-105/comp_lib/linux32/libiomp5.so: undefined reference to `pthread_atfork'

collect2: ld returned 1 exit status

Pavel

0 Kudos
Francesco_Lamonica
974 Views
Hi Pavel,
here is the result with the changed code:
vm-etch-dev32# cat ipp_test_intel_get_procs.cpp
#include
extern "C" {
extern int omp_get_num_procs (void);
}
int main(int argc, char** argv)
{
int p = omp_get_num_procs();
std::cout << "testing ipp: no of procs is" << p << std::endl;
return 0;
}
vm-etch-dev32# g++ -Wall -g ipp_test_intel_get_procs.cpp -L./ -liomp5 -o ipp_test_get_procs
etch:/tmp/ipp-illegal-instruction-test# ./ipp_test_get_procs
Illegal instruction
so it seems that calling omp_get_num_procs triggers the illegal instruction problem (on 7.0.7)
as per the libpthread, i don't know why you get that error, g++ automatically links libpthread on my machines:
etch:/tmp/ipp-illegal-instruction-test# ldd ./ipp_test_get_procs
linux-gate.so.1 => (0xb7fe8000)
libiomp5.so => /tmp/ipp-illegal-instruction-test/libiomp5.so (0xb7f2a000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7e41000)
libm.so.6 => /lib/tls/libm.so.6 (0xb7e1c000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7e11000)
libc.so.6 => /lib/tls/libc.so.6 (0xb7cdf000)
libpthread.so.0 => /lib/tls/libpthread.so.0 (0xb7ccc000)
libdl.so.2 => /lib/tls/libdl.so.2 (0xb7cc8000)
/lib/ld-linux.so.2 (0xb7fe9000)
0 Kudos
Pavel_B_Intel1
Employee
974 Views
Hi Francesco,

so the problem is in libiomp5 library - it is Intel compiler OpenMP library and we will address the problem to the compiler. Thank you very much for your help!!!

To avoid the problem right now I recommend you to use not threaded variant of IPP libraries: link your application with libipp{domain}_l.a libraries - they don't depend on libiomp5 library, I hope it will be ok for you.

Pavel
0 Kudos
pbkenned1
Employee
974 Views
Hello Francesco,
It's not clear to me if your Geode processor supports SSE and/or MMX instructions. If it does not, you might try telling the linker(via gcc) that you're compiling for only a Pentium classtarget, by using
-march=pentium.

Do you still see the error if you add '-m32 -march=pentium' to your compilation command?

I don't have a Geode processor system, but on a Pentiumsystem using g++ 4.1, the program runs normally. I also added '-m32' to generate 32-bit code, as the default on my system is to generate 64-bit code. You shouldn't need that on your 32-bit system (but it can't hurt).

> g++ --version g++ (GCC) 4.1.0 (SUSE Linux)

> g++ U105439.cpp -liomp5 -m32 -march=pentium && ./a.out

testing ipp: no of procs is 2

>

Thanks,
Patrick Kennedy
Intel Developer Support

0 Kudos
Francesco_Lamonica
974 Views
Hi Patrick,
i will do the test you mentioned, however... if the problems lies in the libiomp5.so library and not in my code, how does that affect the code produced by gcc? does the linker inform somehow the shared lib about the architecture?
thanks
-- Add --
i did the test, nothing changed at all
0 Kudos
Reply