Intel® C++ Compiler
Community support and assistance for creating C++ code that runs on platforms based on Intel® processors.
7944 Discussions

Postgresql 9 beta3 compiled (ipo) with Icc (64)

aazue
New Contributor I
977 Views
Hi
I have build last Postgresql (9 beta3) with using -fast(ipo) and xiar
I have tested different options flag (shared-intel and other ..) but always resulting static
binary. Also with ipo require 2 hours,without less that 3 minutes.
with -fast excepted initdb that wrong, all other binary included working
Is not really problem, process initdb as origin icc without ipo or gnu compiler
can be used only to this task. after binary with ipo can be used.
Without ipo flag GNU and Icc result approximatively same performance
used standard Intel Core4 I must test i7 950 when i have to my hand
difficult for search problem, with 2 hours required each step (with ipo)

If ipo used ,to add other external lib backend cpp require also to use xiar to he
work (if caller backend inserted origin icc binary.) (strange....)
( More complex to understand all clear ,with an compiler that is not completely autonomous)
Without this remark are small problem; Icc or Gnu build new Postgresql 9 results sure ,extraordinary
performance ,
Even this two options not really comparable the answers of doubt to substituted choice option NoSql
have no waited...
Regards
0 Kudos
5 Replies
Brandon_H_Intel
Employee
977 Views
-fast implies -static, so if you don't want a static binary, don't use -fast. Use -ipo -O3 -no-prec-div -xHOST -shared or whatever works best.

That build time is extreme, but you should expect some decent increase in build time from using IPO. Try using -ipoN where N is a number. Try 2 first, then try a fairly large number like 100 and see if it helps your build times.

If you have further questions or I didn't address something you mentioned above, let us know.
0 Kudos
aazue
New Contributor I
977 Views
Hi and thank for your answer.
I know that -fast include default static i have use only ipo separate
I must repeat other tests ,probably i have cut and paste wrong cflags ? but strange several times,
or i have select the false shell. (i use( mrxvt with six dozen sessions...). also probably I have modify
Makefile with fault for add two external backend lib..

I have not yet i7 to my hand , i have an other 2 cores mounted ready, i hope that he not require
4 hours,
About time (ipo):
After 2 minutes memory is full time and half size swap used also. hard drive running full all
time less remaining.
I make test with number same you have wrote for see and i answer to you resulting.
Postgres already work nice with ipo static just i want resolve problem of (initdb) that
not working with ipo.

(times make check ) process require dummy database created , only is same i can compare
easily reference improve with and without ipo.

Remark: build Icc Pg 9 without ipo work all perfectly (make check & initdb used correctly)
Regards
0 Kudos
aazue
New Contributor I
977 Views
Hi
With -ipo2 exactly same without (top show ipo or ipo2 working exactly same all)
I have also use -ipo-separate -ipo-jobs4

top show 4 mcpcom and successively several pdflush
volatile memory is used full and now all size parted swap used.
Size memory is not divided equal each, process mcpcom, always, rotate 1 have large
and 3 other the bread crumbs....
Result is more catastrophic ... 3 hours passed and not ended .. i think
better I stop process before that HD is killed...
I think to merge (make --job=4) with (ipo-jobs4) wrong way....
same when you have the mother of your wife in your house also....
Better I wait machine CoreI7 before to extend..

Also Ipo is just an option...
To end side more positive and without objective to compare GNU compiler with ICC
I think that tests is given after demonstrated how new programming accorded with the new hardware
can be nice.
I hope ,as prove, all negative people for the change understand that new programming and new hardware
perfectly justified and controlled by programmers..

Tests Postgresql 8.4.3 compared Postgresql 9 beta3 ICC and GCC (#2 Core Intel. 1.8 GHz).

make distclean
LANG=C; export LANG
CC="/opt/intel/Compiler/11.1/072/bin/intel64/icc"; export CC
LD_LIBRARY_PATH=/opt/intel/Compiler/11.1/072/lib/intel64;export LD_LIBRARY_PATH;export LD_LIBRARY_PATH
CFLAGS="-O3 -lpthread -m64 -axSSE3 -parallel -march=core2 -mtune=core2 -Wpointer-arith -fno-strict-aliasing "; export CFLAGS
./configure
time make --jobs=4
time make check

Postgresql 8.4.3 with ICC process time make --job=4

make[2]: Leaving directory `/usr/src/download/postgresql-8.4.3/src/test/regress'
make[1]: Leaving directory `/usr/src/download/postgresql-8.4.3/src'
make -C config all
make[1]: Entering directory `/usr/src/download/postgresql-8.4.3/config'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/usr/src/download/postgresql-8.4.3/config'
All of PostgreSQL successfully made. Ready to install.

real 5m59.203s
user 9m17.895s
sys 0m35.270s

Postgresql 8.4.3 with ICC process time make check
=======================
All 120 tests passed.
=======================

make[2]: quittant le rpertoire /usr/src/download/postgresql-8.4.3/src/test/regress
make[1]: quittant le rpertoire /usr/src/download/postgresql-8.4.3/src/test

real 0m37.792s
user 0m9.181s
sys 0m9.973s
postgres@debian-bustaf:/usr/src/download/postgresql-8.4.3$


Postgresql 9 beta3 with ICC process time make --job=4

make[2]: Leaving directory `/usr/src/download/postgresql-9.0beta3_icc/src/test/regress'
make[1]: Leaving directory `/usr/src/download/postgresql-9.0beta3_icc/src'
make -C config all
make[1]: Entering directory `/usr/src/download/postgresql-9.0beta3_icc/config'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/usr/src/download/postgresql-9.0beta3_icc/config'
All of PostgreSQL successfully made. Ready to install.

real 6m17.968s
user 9m40.560s
sys 0m36.638s


Postgresql 9 beta3 with ICC process time make check

=======================
All 122 tests passed.
=======================

make[2]: quittant le rpertoire /usr/src/download/postgresql-9.0beta3_icc/src/test/regress
make[1]: quittant le rpertoire /usr/src/download/postgresql-9.0beta3_icc/src/test

real 0m26.765s
user 0m5.708s
sys 0m3.612s
postgres@debian-bustaf:/usr/src/download/postgresql-9.0beta3_icc$


/////////////////////////////////////////////////////
make distclean
LANG=C; export LANG
CC="gcc"; export CC
CFLAGS="-O3 -m64 -pthread -ftree-vectorize -march=core2 -mtune=core2 -fomit-frame-pointer -pipe" ; export CFLAGS
./configure
time make --jobs=4
time make check

Postgresql 8.4.3 with GCC process time make --job=4

make[2]: Leaving directory `/usr/src/download/postgresql-8.4.3/src/test/regress'
make[1]: Leaving directory `/usr/src/download/postgresql-8.4.3/src'
make -C config all
make[1]: Entering directory `/usr/src/download/postgresql-8.4.3/config'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/usr/src/download/postgresql-8.4.3/config'
All of PostgreSQL successfully made. Ready to install.

real 2m44.693s
user 4m41.530s
sys 0m20.513s
debian-bustaf:/usr/src/download/postgresql-8.4.3#


Postgresql 8.4.3 with GCC process time make check
=======================
All 120 tests passed.
=======================

make[2]: quittant le rpertoire /usr/src/download/postgresql-8.4.3/src/test/regress
make[1]: quittant le rpertoire /usr/src/download/postgresql-8.4.3/src/test

real 0m37.553s
user 0m8.961s
sys 0m10.125s
postgres@debian-bustaf:/usr/src/download/postgresql-8.4.3$

Postgresql 9 beta3 with GCC process time make --job=4
make[2]: Leaving directory `/usr/src/download/postgresql-9.0beta3_gnu/src/test/regress'
make[1]: Leaving directory `/usr/src/download/postgresql-9.0beta3_gnu/src'
make -C config all
make[1]: Entering directory `/usr/src/download/postgresql-9.0beta3_gnu/config'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/usr/src/download/postgresql-9.0beta3_gnu/config'
All of PostgreSQL successfully made. Ready to install.

real 2m51.893s
user 4m53.606s
sys 0m20.389s
debian-bustaf:/usr/src/download/postgresql-9.0beta3_gnu#


Postgresql 9 beta3 with GCC process time make check
=======================
All 122 tests passed.
=======================

make[2]: quittant le rpertoire /usr/src/download/postgresql-9.0beta3_gnu/src/test/regress
make[1]: quittant le rpertoire /usr/src/download/postgresql-9.0beta3_gnu/src/test

real 0m27.274s
user 0m5.500s
sys 0m3.540s
postgres@debian-bustaf:/usr/src/download/postgresql-9.0beta3_gnu$


********************
Example result when i use initdb manually same required with make check (option icc with flag ipo and xiar used)


postgres@debian:/usr/local/pgsql/bin$ ./initdb -D /home/pgsql
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale fr_FR.
The default database encoding has accordingly been set to LATIN1.
The default text search configuration will be set to "french".

fixing permissions on existing directory /home/pgsql ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 24MB
creating configuration files ... ok
creating template1 database in /home/pgsql/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating conversions ... FATAL: could not load library "/usr/local/pgsql/lib/ascii_and_mic.so": /usr/local/pgsql/lib/ascii_and_mic.s
o: undefined symbol: pg_ascii2mic
STATEMENT: CREATE OR REPLACE FUNCTION ascii_to_mic (INTEGER, INTEGER, CSTRING, INTERNAL, INTEGER) RETURNS VOID AS '$libdir/ascii_and
_mic', 'ascii_to_mic' LANGUAGE C STRICT;
child process exited with exit code 1.

Precision: Pg not require obligatory the big hardware.
Intel Core 2 is already very nice for an database Pg operational
I speak about of I Core 7 only to process compile with flag ipo.

Regards

0 Kudos
aazue
New Contributor I
977 Views
Hi
I extended to double the memory same machine Intel 2 core
I have used xterm shell
I have used make without (--jobs) and flag ipo2 -static-intel (static only one not working)
Time build is reduced to 35 minutes.
Now i have static binary, but same problem initdb persisting ...
I have used init of origin build by icc without ipo for initialize an repertory database.
now as new database created , i going to directory resulting (make install of build with ipo )
./pg_ctl -D /home/pgsql start
engine (have ipo) start database without problem

I can use now command make installcheck
with installcheck the database already started is used for test without initdb necessary
The result with binary have ipo flag, is wrong, half number of test passed failed...

All problems are detailed this two files
(repertory)/src/test/regress/regression.out
(repertory)/src/test/regress/regression.diffs

(I can add this two file attached if have interest for an user)

I suspect problem is (xiar / xild) process.. with libgcc_s.so ,that impact also lib postgresql
Strange that is working perfectly builded all only if without ipo flag....

That he already working compiled Icc is the more important..


Regards..

0 Kudos
aazue
New Contributor I
977 Views
Hi
-ip (ipo separate each file) (only shared)
It work small better.
I have tested also (flto (ipo gnu compiler))
same Icc .
I have add some flag (ClooG side) for improve better.
ICC and GCC Improve are no demonstrated (with time make check)
I have made test with the very great tasks with my source.
New engine Postgres 9 b4 is very very good.
I have also build with icc & Gcc (ipo & lto) last Apache 2.3.6-alpha
It work perfectly...

over more of 200/300 functions (large) , not problem observed.
With lto you must verify that you having source correctly wrote.
Backend with Icc and Gcc work also perfectly two side.(ipo & lto)
(Icc -fast) or (-fwhole-program Gcc) not working for build engine
but working in program backend.

I have I7 for test but is custumer machine if I install beta program
I must renew all the install machine.
I wait buy for personal, but price 950 have not yet less.
Nb
I have build last Gcc 4.5.1 with all option without problem.
It work also very well; i must learn about ClooG for improve better ...
I have search where to download last icc 12 (beta)(Linux) but without
success...

Regards
0 Kudos
Reply