topic Creating and using an IPP-based static library in Intel® Integrated Performance Primitives

Creating and using an IPP-based static library

apolo74 — Tue, 09 Nov 2010 13:28:12 GMT

Hi everybody,

I'm sorry for opening a new thread about this issue but I'm very confused with the other threads and documents I've been reading about linking IPP. I really need some help with this as clear and detailed as possible.

I have created a class of simple math functions (sigmoidal, gaussians for 1D and for 2D vectors, exponentials, etc.) using IPP. I would like that people in my lab starts using this collection of functions so I decided to create a library. From what I understand, I should create a static library because I'm the only one with the IPP libraries installed in my computer. The idea then is that I pack all IPP functions and definitions needed for running my own functions within a static library and give it to my colleagues. They do NOT have IPP in their computers and shouldn't need to have it since my library should contain everything, right???

This is how I understand the whole process: First, build a library; Second, use the library.

Building the library:

- I took the files from "advanced-usage > linkage > mergedlib" to create my own library. I just modified the file called funclist.h where I put those IPP functions that I'm using in my class.

- I then included the file called ippmerged.h in the header of my class. (Before this, my class header included only the file ipp.h )

- Finally I compiled my class linking with the following libraries: libippsemerged.a libippsmerged_t.a libippvmemerged.a libippvmmerged_t.a libippcore_t.a libiomp5.a
- I got a flawless compilation, the library was created without any errors or warnings.

Using the library:

- I went to a different computer, copied my library and the following headers: myclass.h, funclist.h and ippmerged.h

- I created a very simple main.cpp file with the following lines:

//---------------------------------------------------------------------

#include

#include "myclass.h"

#include "ippmerged.h"

using namespace std;

int main( int argc, char *argv[] ){

if( InitStatic() ){

const IppLibraryVersion* libver; // library info structure ptr

// first print the version of the libraries

libver = ippsGetLibVersion();

printf( " %s %s \\n", libver->Name, libver->Version );

cout << "byebye world" << endl;

return 0;

}

return -1;

}

//---------------------------------------------------------------------

- Finally, I compiled this file including the following libraries: libmyclass.alibippsemerged.a libippsmerged_t.a libippvmemerged.a libippvmmerged_t.a libippcore_t.a libimf.a libsvml.a libirc.a libiomp5.alibpthread.a

- I get an error saying it can't find ipp.h (error: ipp.h: No such file or directory), from then on I assume the errors are related to not having the definitions from ipp.h.

Is there something wrong in what I'm doing??? or am I supposed to copy all the IPP headers into the rest of the computers together with my library??? Thanks for any help and suggestions,

Boris

PS: I followed this instructions fromhttp://software.intel.com/en-us/articles/performance-tools-for-software-developers-using-intel-ipp-threaded-static-libraries/. And from the IPP documentation.

PSS: I'm working with Ubuntu 10.04, Intel Compiler 11.1 (073), the output ofippsGetLibVersion() is

ippspxt.a 6.1 build 137.56

Creating and using an IPP-based static library

Vladimir_Dudnik — Tue, 09 Nov 2010 17:38:02 GMT

Hello,

when you see compiler's message that it can't find or open some particular file (ipp.h in your case), please treat it just as it is. Some of your source files does include that header file (namely, contain text like this: #include or #include "ipp.h"). If you do not have ipp.h file on your second computer then compiler will not be able to find this file.

Regards,
Vladimir

Creating and using an IPP-based static library

apolo74 — Wed, 10 Nov 2010 08:25:17 GMT

Hi Vladimir and thanks for replying,

I know that I am including ipp.h inside myclass.h, but from what I understand I can't avoid that step. I have to include myclass.h in any new application that I want to create on a second computer; and MYCLASS needs to include ipp.h to define all IPP variables and functions. So, what you are saying is that I have to includeall IPP headersin the second computer???

Boris

Creating and using an IPP-based static library

Vladimir_Dudnik — Wed, 10 Nov 2010 19:00:05 GMT

Obviously you do not need declaration of functions or data types which you do not use in your application. The ipp.h file (if you have looked into it) is just auxiliary header which combines all IPP header files into one place. If you only use, say ippsCopy_8u function, then you can extract declaration of this function from ipps.h file and include only this part into your myclass.h file. If you would like to avoid tricks with extraction declarations of separate functions then you may want to provide with your application only ipps.h file (for example mentioned above).

Vladimir

Creating and using an IPP-based static library

apolo74 — Thu, 11 Nov 2010 12:21:55 GMT

Hi again Vladimir,

thanks for the explanation, I did as you said but using the example given in "ipp-samples > advanced-usage > linkage > customso". In that example I'm allowed to make a list of the IPP functions that I want to encapsulate in a .so file for later "transport". I've tested this in different computers and it is working.

I just would like to ask you about efficiency of these two methods... should I try to create a mergedlib instead of a customso library? I don't think size of the final library is a problem for me or the people that will be using my class. The important part is which method offers me the best performance.

Boris

Creating and using an IPP-based static library

Vladimir_Dudnik — Thu, 11 Nov 2010 22:16:51 GMT

Hi Boris,

non threaded merged static library and non threaded custom shared library shouldhave approximately the same performance. They both build from the same IPP binary code.

Regards,
Vladimir

Creating and using an IPP-based static library

apolo74 — Fri, 12 Nov 2010 13:39:32 GMT

Thanks for the clarification, I just realized that the build32.sh script was supposed to create both the customso non-threaded and the threaded version. The non-threaded .so is generated and working in other computers without problems but (after fixing the paths to the right 'lib' directory) I get the following error when trying to generate the threaded version:

ld: /usr/lib/libpthread_nonshared.a(pthread_atfork.oS): relocation R_386_GOTOFF against undefined hidden symbol `__dso_handle' can not be used when making a shared object

I really don't know what this mean or where to look to fix this part, do you have any idea what this mean? and how to fix it? or in the last case, how can I generate a threaded library?

Boris

Creating and using an IPP-based static library

Chao_Y_Intel — Mon, 15 Nov 2010 01:10:49 GMT

Hi Boris,

The error showed the static pthread library (libpthread.a) could not be linked to created an shared object.

thread libraries is often linked dynamically, changing it tolibpthread.so work for you?

Thanks,
Chao

Creating and using an IPP-based static library

apolo74 — Wed, 24 Nov 2010 11:06:00 GMT

Hi Chao,

sorry for the late reply, I understand your point but I'm too newbie with Makefiles and can't find how to change specific parts so that I can get what I want. I will be very grateful if you or someone else give me a hand with this, as I said before I'm just following and trying to modify the files from ipp-samples/advanced-usage/linkage/customso. The file build32.sh calls the Makefile with:

make -f Makefile ARCH=${SAMPLE_ARCH} COMP=${SAMPLE_COMP} CC=${CC} CXX=${CXX} LINKAGE=${LINKAGE} IPPROOT=${IPPROOT} && SAMPLE_STATUS="-OK"

[ "$SAMPLE_STATUS" != "-FAILED" ] && make -f Makefile ARCH=${SAMPLE_ARCH} COMP=${SAMPLE_COMP} CC=${CC} CXX=${CXX} LINKAGE=${LINKAGE} IPPROOT=${IPPROOT} THR_SFX="_t" && SAMPLE_STATUS="-OK"

Here is the Makefile:

INTELROOT := /opt/intel/Compiler/11.1/073/lib/ia32

TARGDIR := ./bin

LIBNAME := $(TARGDIR)/libippFuncs$(THR_SFX).so

TSTNAME := $(TARGDIR)/testTools$(THR_SFX)

EXPORT := export.def

SCRIPT := library.script

IPPLIB := ippiemerged ippimerged$(THR_SFX) ippsemerged ippsmerged$(THR_SFX) ippvmemerged ippvmmerged$(THR_SFX) ippcore$(THR_SFX)

LINK := ld

ifdef THR_SFX

ILIB := $(addsuffix .a,$(addprefix $(INTELROOT)/lib,svml imf irc irc_s iomp5))$(LIBPTHREAD)

endif

#all : prepare build_lib build_test run

all : prepare build_lib

build_lib : $(LIBNAME)

$(LIBNAME) :$(SCRIPT) $(TARGDIR)/init.o

$(LINK) -shared -zdefs -o $@ $^ -L$(ICCLIBPATH) -L$(IPPROOT)/lib -L$(TARGDIR) $(addprefix -l, $(IPPLIB)) $(ILIB) -lm -lc

$(TARGDIR)/init.o : init.c

$(CC) -c -fpic -I$(IPPROOT)/include -o $@ $^

$(SCRIPT) : $(EXPORT)

perl script.pl $< >$@

...

From what I understand, the same line is being used to build both versions ( $(LINK) -shared ... ), should it be "shared" for both versions???As it was in my previous message, the non-threaded version is built without problems and it is working in other computers. The Threaded version is not being built and the output goes like this:

+----------------------------------------------------------------------------+

|Test : intel build Wed Nov 24 11:22:34 AM 2010

|Hostname : boris-laptop

|Function : no

|Description : building intel sample

|Class : Information

|Source : /home/boris/cppStuff/myStuff/DFT/intel

|Executable : build32.sh

+----------------------------------------------------------------------------+

Found compiler icc111 ( autodetect ) is /opt/intel/Compiler/11.1/073

Compiler environment is loaded

IPP environment found at /opt/intel/Compiler/11.1/073/ipp/ia32/tools/env/ippvars32.sh

IPP environment is loaded

All necessary packages and requred compilers are checked

Environment is setup. Build process is started. Please wait build results ...

rm -f ./bin/testTools

rm -f ./bin/libippFuncs.so

rm -f library.script

rm -rf ./bin

rm -f *.o

perl script.pl export.def >library.script

/opt/intel/Compiler/11.1/073/bin/ia32/icc -c -fpic -I/opt/intel/Compiler/11.1/073/ipp/ia32/include -o bin/init.o init.c

ld -shared -zdefs -o bin/libippFuncs.so library.script bin/init.o -L/opt/intel/Compiler/11.1/073/lib/ia32 -L/opt/intel/Compiler/11.1/073/ipp/ia32/lib -L./bin -lippiemerged -lippimerged -lippsemerged -lippsmerged -lippvmemerged -lippvmmerged -lippcore -lm -lc

ld -shared -zdefs -o bin/libippFuncs_t.so library.script bin/init.o -L/opt/intel/Compiler/11.1/073/lib/ia32 -L/opt/intel/Compiler/11.1/073/ipp/ia32/lib -L./bin -lippiemerged -lippimerged_t -lippsemerged -lippsmerged_t -lippvmemerged -lippvmmerged_t -lippcore_t /opt/intel/Compiler/11.1/073/lib/ia32/libsvml.a /opt/intel/Compiler/11.1/073/lib/ia32/libimf.a /opt/intel/Compiler/11.1/073/lib/ia32/libirc.a /opt/intel/Compiler/11.1/073/lib/ia32/libirc_s.a /opt/intel/Compiler/11.1/073/lib/ia32/libiomp5.a -L/opt/intel/Compiler/11.1/073/lib/ia32 -liomp5 -lpthread -lm -lc

/opt/intel/Compiler/11.1/073/lib/ia32/libiomp5.a(kmp_sched.o): In function `void __kmp_for_static_init(ident*, int, int, int*, unsigned long long*, unsigned long long*, traits_t::signed_t*, traits_t::signed_t, traits_t::signed_t)':

../../src/kmp_sched.cpp:(.text+0x2c6): undefined reference to `__umoddi3'

../../src/kmp_sched.cpp:(.text+0x484): undefined reference to `__umoddi3'

../../src/kmp_sched.cpp:(.text+0x59e): undefined reference to `__umoddi3'

/opt/intel/Compiler/11.1/073/lib/ia32/libiomp5.a(kmp_sched.o): In function `void __kmp_for_static_init(ident*, int, int, int*, long long*, long long*, traits_t::signed_t*, traits_t::signed_t, traits_t::signed_t)':

../../src/kmp_sched.cpp:(.text+0x9c6): undefined reference to `__umoddi3'

../../src/kmp_sched.cpp:(.text+0xb9a): undefined reference to `__umoddi3'

/opt/intel/Compiler/11.1/073/lib/ia32/libiomp5.a(kmp_sched.o):../../src/kmp_sched.cpp:(.text+0xcbf): more undefined references to `__umoddi3' follow

/opt/intel/Compiler/11.1/073/lib/ia32/libiomp5.a(kmp_dispatch.o): In function `void __kmp_dispatch_init(ident*, int, sched_type, long long, long long, traits_t::signed_t, traits_t::signed_t, int)':

../../src/kmp_dispatch.cpp:(.text+0x6352): undefined reference to `__moddi3'

ld: /usr/lib/libpthread_nonshared.a(pthread_atfork.oS): relocation R_386_GOTOFF against undefined hidden symbol `__dso_handle' can not be used when making a shared object

ld: final link failed: Bad value

make: *** [bin/libippFuncs_t.so] Error 1

+----------------------------------------------------------------------------+

|-OK intel Wed Nov 24 11:22:35 AM 2010

+----------------------------------------------------------------------------+

This is already a very long message, I'm sorry for that but I think I need to put everything if I want to show the whole picture of my problem. I hope you guys can give me some help, thanks.

Boris

Creating and using an IPP-based static library

Ying_H_Intel — Mon, 10 Jan 2011 03:12:26 GMT

Hello Boris,

About first issue "/usr/lib/libpthread_nonshared.a(pthread_atfork.oS): relocation R_386_GOTOFF":
it seems had been discussed in http://software.intel.com/en-us/forums/showthread.php?t=70367

The problem mainly arethestatic intel OpenMP threading library libiomp5.a andthestatic system threading library libpthread.a.
the Ubuntu bugs tracker: https://bugs.launchpad.net/ubuntu/+source/gcc-defaults/+bug/358665
the forum 70367thread hints, the issue can be solved - there was wrong libpthread.so library and it solved by upgraged pthread libraries under Redhat.

Additionation, thedynamic(shared) libiomp5.so (in ia32 and em64t) are recommended tobe used for any openmp threaded application, I'd like suggestto changeMakefile to make surelink bothdynamic libiomp5.so and libpthread.so

ifdef THR_SFX

ILIB := $(addsuffix .a,$(addprefix $(INTELROOT)/lib,svml imf irc irc_s))-liomp5
endif

and $(TSTNAME) : $(LIBNAME)
$(CC) -o $@ -I$(IPPROOT)/include ipptwo.c -L. -L$(TARGDIR) -lusr$(THR_SFX) -lpthread -lm -lc.

I update the build32_t.sh and Makefile.32 here.
You can copy them to the customso directory
enter command
>chmod +x build32_t.sh
>./build32_t.sh

Hope it helps,
Ying

Please use the latest version

Yuan_X_Intel — Wed, 07 Mar 2018 09:20:37 GMT

Please use the latest version IPP.

For my case, i can find the ipp*.h in /opt/intel/compilers_and_libraries2018/linux/ipp/include. And remember to execute "source /opt/intel/compilers_and_libraries2018/linux/bin/compilervars.sh" before use "make -f Makefile_lin".