Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Bo_Jensen
Beginner
594 Views

How to minimize size of a shared library

I am building a large shared library, which uses a lot of boost functionality.

The library files are being compiled with :

[bash]-fPIC -fvisibility=hidden -fvisibility-inlines-hidden -ffunction-sections -fdata-sections [/bash] and linked with :

[bash]-static-libgcc -ldl -Wl,--gc-sections[/bash] The problem is the library size blow up to a amazing 17MB, without the above settings it's 28MB.. If I do a strip on the library afterwards, we get down in about 13-14MB, which is still too much. Looking at the output of nm it seems the hidding does not work properly, since it's exporting a large number of functions still.

With exact the same settings I can get 5.5MB after strip with gcc and 9MB before strip.

Even if I compile with intel and link with gcc directly, then the problem persists.

What settings do I need to set for intel to remove all the redudancy ?
0 Kudos
10 Replies
xunxun
Beginner
594 Views

Try -O1 or -Os?

ipo sometimes can reduce the size, and sometimes may increase the size.
Bo_Jensen
Beginner
594 Views

Thanks for the suggestion, I had already tried it. With -Os the library gets down in 15MB unstripped, which is still far from gcc size and important optimization is now disabled, since it's a libary doing numerical calculations I don't want to lower the optimization level.

The problem is that it exports a huge number of functions, which is not marked to be exported.
SergeyKostrov
Valued Contributor II
594 Views

Quoting Bo Jensen
...The problem is that it exports a huge number of functions, which is not marked to be exported...


I think you need to review a couple of cases when some functions, declared as non-exported, are included in
a binary file of the shared library. I would consider cases:

1. If some 'FunctionA' is declared as inline, has implementation in a header file, and called at least
once in your codes onlythenithas to be included in a binary file ( this is expected result )

2.If some 'FunctionB' is declared as inline, has implementation in a header file, and not called at all in
any codes then it doesn't have to be included in a binary file ( this is expected result ).If it is included
then it looks like there is a problem with a C/C++ compiler.

The Boost is a huge template library andasimpleC++ templates-based test-case that uses atest shared
library, with some set offunctions declared differently,could helpto understand the problem.

xunxun
Beginner
594 Views

Can you ldd your library?
May consider that make some other library dynamic link.

ps: in general, the size icc generated is larger than gcc.
Bo_Jensen
Beginner
594 Views

Have not had time to look at this until now.

Try to use this as an example :

[bash]#include #include #include template class __attribute__ ((visibility ("hidden"))) MyDummyClass { private: std::map<:STRING> freq; public: void MyDummyClassFunc() { std::string word; while (std::cin >> word) { freq[word]++; } std::map<:STRING>::const_iterator iter; for (iter=freq.begin(); iter != freq.end(); ++iter) { std::cout << iter->second << " " << iter->first << std::endl; } } }; int Dummy() { MyDummyClass test; test.MyDummyClassFunc(); return 0; }
[/bash] Compiled with intel :

[bash]icpc -shared test.cpp -fvisibility=hidden -fvisibility-inlines-hidden -ffunction-sections -fdata-sections -Wl,--gc-sections -o test.so [/bash] and similar for g++.


On the intel created binary I get :

[bash]00002140 T .text._ZN12MyDummyClassIiED1Ev 000020d0 T .text._ZNSt4pairIKSsiED1Ev 00001e70 T .text._ZNSt8_Rb_treeISsSt4pairIKSsiESt10_Select1stIS2_ESt4lessISsESaIS2_EE11_M_put_nodeEPSt13_Rb_tree_nodeIS2_E 00001e80 T .text._ZNSt8_Rb_treeISsSt4pairIKSsiESt10_Select1stIS2_ESt4lessISsESaIS2_EE16_M_insert_uniqueERKS2_ 000020f0 T .text._ZNSt8_Rb_treeISsSt4pairIKSsiESt10_Select1stIS2_ESt4lessISsESaIS2_EE8_M_eraseEPSt13_Rb_tree_nodeIS2_E 00003c30 T __intel_f2int 00002500 T __intel_new_proc_init 00002600 T __intel_new_proc_init.A 00002540 T __intel_new_proc_init.H 00003210 T __intel_new_proc_init_B 000034d0 T __intel_new_proc_init_B.A 00003410 T __intel_new_proc_init_B.J 00003340 T __intel_new_proc_init_B.K 00003270 T __intel_new_proc_init_B.L 000027e0 T __intel_new_proc_init_G 00002880 T __intel_new_proc_init_G.A 00002820 T __intel_new_proc_init_G.R 000028d0 T __intel_new_proc_init_H 00002970 T __intel_new_proc_init_H.A 00002910 T __intel_new_proc_init_H.P 000026f0 T __intel_new_proc_init_I 00002790 T __intel_new_proc_init_I.A 00002730 T __intel_new_proc_init_I.S 000029f0 T __intel_new_proc_init_L 00002b00 T __intel_new_proc_init_L.A 00002a30 T __intel_new_proc_init_L.O 00003720 T __intel_new_proc_init_N 000039e0 T __intel_new_proc_init_N.A 00003920 T __intel_new_proc_init_N.J 00003850 T __intel_new_proc_init_N.K 00003780 T __intel_new_proc_init_N.L 00002f90 T __intel_new_proc_init_P 000030a0 T __intel_new_proc_init_P.A 00002fd0 T __intel_new_proc_init_P.L 00002b80 T __intel_new_proc_init_S 00002c90 T __intel_new_proc_init_S.A 00002bc0 T __intel_new_proc_init_S.N 00002d10 T __intel_new_proc_init_T 00002e20 T __intel_new_proc_init_T.A 00002d50 T __intel_new_proc_init_T.M 00002610 T __intel_proc_init 000026e0 T __intel_proc_init.A 00002650 T __intel_proc_init.H 00003550 T __intel_proc_init_B 000036a0 T __intel_proc_init_B.A 00003610 T __intel_proc_init_B.J 000035e0 T __intel_proc_init_B.K 000035b0 T __intel_proc_init_B.L 00003a60 T __intel_proc_init_N 00003bb0 T __intel_proc_init_N.A 00003b20 T __intel_proc_init_N.J 00003af0 T __intel_proc_init_N.K 00003ac0 T __intel_proc_init_N.L 00003120 T __intel_proc_init_P 00003190 T __intel_proc_init_P.A 00003160 T __intel_proc_init_P.L 00002ea0 T __intel_proc_init_T 00002f10 T __intel_proc_init_T.A 00002ee0 T __intel_proc_init_T.M 000024b0 T __intel_set_fpx_mask 00002490 T __intel_set_fpx_mask.A 00002460 T __intel_set_fpx_mask.H 000040bc T _fini 00001c9c T _init [/bash] While on the g++ generated I get the expected :

[bash]00003fdc T _fini 00002e0c T _init [/bash] So notice that the map stl functions gets exported ! I realize there are some issues with exporting STL functions, but gcc seem to manage that fine.

Now from the example it seems it's only related to STL, but it's not, I see some really strange exports, which I have a hard time reproducing in a small example.

I should say how I am building my library : I have module sources some with template functions and some with out. In some cases the templates are explicit instantiated, so I can reduce compile time and build in a modular fashion. All my classes have a hidding attribute and the explicit instantiations too. The problem is if a class get's implicit instantiated before explicit instantiation, then the attribute hiding is not controlled and it get's exported.

The final build is a c library, that is build with a wrapper class.

So my question is how can I control which functions get's exported, especially for external template classes like boost or STL ?
JenniferJ
Moderator
594 Views

I thought that I should be able to duplicate your issue with the testcase attached. But no. I tried 12.1 (Version 12.1.3.293 Build 20120212)in IA32 Ubuntu 11.04and SUSE Linux Server 11 x64, get the following.

32053 2012-03-29 09:57 t_gcc_ia32.so

30388 2012-03-29 09:56 t_icc_ia32.so

43273 2012-03-29 09:51 t_gcc_x64.so

15513 2012-03-29 09:50 t_icc_x64.so

cmd used is from your posting with "-fPIC": icpc -fPIC -shared t.cpp -fvisibility=hidden -fvisibility-inlines-hidden -ffunction-sections -fdata-sections -Wl,--gc-sections -o t.so

So I might have missed something. Please let me know if the above is the what you have. We'd like to improve the code size issue if possible.

Thanks,
Jennifer

Bo_Jensen
Beginner
594 Views

Thanks for looking into this.

I might not have explained it clear enough, but above example is just a small one illustrating that exports end up in the library, which is not intended and most certainly different from gcc. So I was not argumenting on library size on this example only the exports. I think what goes on for my large library is that soo many functions get's exported, which makes the linker command --gc-section useless, since it can not tell which ones are not needed. Since my code is huge, I thought starting with having similar exported function like gcc was a good starting point. Try to run nm test.so | grep " T " on your two libraries.
JenniferJ
Moderator
594 Views

I see.
With the "grep" cmd, I did see the big difference in the symbols being exported.

Let me file a bug to the compiler about this issue, but I'm not sure how likely it can be addressed in the current release though.

thanks,
Jennifer

Bo_Jensen
Beginner
594 Views

I have now tried everything I could come up with, but the only way I can get the expected exported functions is by using a Version Script.
JenniferJ
Moderator
594 Views

Hello,
This issue has been fixed in the latest update 11.

You can download the update 11 from Intel Registration Center.

Jennifer
Reply