ICC benchmarking and ICC vs GCC

[bouache]

Hi,

as you see I am new here in this forum.

I have a project comparing ICC vs GCC. I am looking for the best way to start this project. any links/ tutorial/ documentations are more than welcome.

thinking to do some benchmarking with both compilers using SPECInt or any other type of workloads/ programs.

Thanks for your help.

-Lx 

[KitturGanesh]

Thanks Steve for responding on Fortran (Mr B, Steve is "the" Fortran Guru, fyi) Well, I don't know if the performance team are willing or can share the config files (need to check). Of course, SPEC publishes the options used by the submitters of benchmark data so you can get that info from their site, fyi. _Kittur

[TimP]

Normally you would use the same compiler for F77 and F90. SPEC entries have config posted.

[KitturGanesh]

BTW, the complete config files should be found on the Spec org site, so you can go to www.spec.org for download of related config files from various sponsors of the benchmarks, fyi.

[bouache]

Thanks for all of you.

I would like to do some optimization using the ICC. which flags you are recommending vs GCC (optimized version as well).

the idea is to have the spec perfromance for GCC vs ICC (baseline), GCC vs ICC (optimized version)?

Thanks for your help

[KitturGanesh]

MrB, well, what you can do is to go to to www.spec.org site and click on the results tab and from there say you can select the CPU2006 Search Form below and configure your advanced search to indicate a value for Compiler matches as: "Version 14.0.0.080 of Intel C++". You'll get a dump of HW vendor, System, Disclosure etc., containing corresponding config file that you can click and view the complete configuration and see what options are used for each of the Spec tests. For baseline, you can try -O2 optimization and see what you get with icc and gcc and you can also see what's used as default in config file for those tests also. Likewise, for optimization, you can again refer to the sort of flags used in the config file for the tests. In general, if you're running on a system with AVX2 for example, you can use for example with icc: -xCORE-AVX2 -ipo -O3 -no-prec-div -opt-prefetch and so on. Like wise, with GCC you can correspondingly use for example: -Ofast-march=core-avx2 -mfpmath=sse -funroll-loops -fasynchronous-unwind-tables and so on..... I think taking a look at the config files for published Spec benchmarks would be the way to go since you're running Spec.... _Kittur

[bouache]

Thanks Kittur.

I guess its the same thing for C and CXX?

ICC

COPTIMIZE       = -xCORE-AVX2 -ipo -O3 -no-prec-div -opt-prefetch
CXXOPTIMIZE     = -xCORE-AVX2 -ipo -O3 -no-prec-div -opt-prefetch

GCC

COPTIMIZE       = -Ofast-march=core-avx2 -mfpmath=sse -funroll-loops -fasynchronous-unwind-tables
CXXOPTIMIZE     = -Ofast-march=core-avx2 -mfpmath=sse -funroll-loops -fasynchronous-unwind-tables

Thanks,

[KitturGanesh]

MrB, yes it's the same for C and C++. The only time you want to use differently is when applications you compile for example use C++ features say then may have to use -std= option where is c++11 etc., during compilation but don't think you have such a scenario in the spec tests you're trying to compile/execute. _Kittur

[bouache]

thanks Kittur.

in my config file I have wrote this and it's not working?! can you help?

for GCC:

#Optimization AVX
int=base=gcc345-yahoo-opt=default:
notes0080=   Baseline C: -Ofast-march=core-avx2 -mfpmath=sse -funroll-loops -fasynchronous-unwind-tables
COPTIMIZE       = -Ofast-march=core-avx2 -mfpmath=sse -funroll-loops -fasynchronous-unwind-tables
notes0080=   Baseline C++: -Ofast-march=core-avx2 -mfpmath=sse -funroll-loops -fasynchronous-unwind-tables
CXXOPTIMIZE     = -Ofast-march=core-avx2 -mfpmath=sse -funroll-loops -fasynchronous-unwind-tables

for ICC

int=base=icc=default:
notes0080=  Baseline C:   icc -xCORE-AVX2 -ipo -O3 -no-prec-div -opt-prefetch
COPTIMIZE       = -xCORE-AVX2 -ipo -O3 -no-prec-div -opt-prefetch
notes0080=  Baseline C++:   icpc -xCORE-AVX2 -ipo -O3 -no-prec-div -opt-prefetch
CXXOPTIMIZE     = -xCORE-AVX2 -ipo -O3 -no-prec-div -opt-prefetch

[KitturGanesh]

MrB, I searched the spec site and I have attached a config file (to this thread, that you can download) used for one of the hw vendor's cpu2006 test run. You can see clearly there how the general default values are set, and then how for each individual unit tests the values are changed accordingly and you can create one on similar lines.

BTW, the attached config file also shows usage of avx2 and other options like settings for CC, CXX, SSE used (avx, avx2, sse4.2 etc) and set proper values for each unit test and override defaults if need be explicitly.   BTW, make sure your system supports avx if you're using avx?  To check if your system supports avx, you can do %cat /proc/cpuinfo and look at the values in the 'flags" output. It will have avx or avx2 depending on the system. Hope this helps...

_Kittur

[TimP]

It looks like you omitted the space preceding -march=.... I've already made suggestions about unrolling options, so I don't expect further comments to have much influence. It doesn't make much sense to benchmark gcc 3.45 (released 7 years ago) when no compilers that old are still supported, unless you hope to show the degree of progress made over those years. I don't think any linux distros based on such an old gcc support any current CPUs. If you would be more explicit about probems you encountered, the responses you get might be more on target.

[KitturGanesh]

Tim, you're correct just noticed gcc345 in MrB's response. MrB, Tim's correct, it's a very old version and would fail miserably on several features/optimizations that present versions of GCC supports. I'd suggest you use at least a version of GCC 4.4 and above for your benchmarking needs.... _Kittur

[bouache]

thanks for your answer

I have the results for GCC, GCC with optimization, ICC and ICC with Optimization flags:

the base rate for SPECInt2000 for GCC = 358, GCC with OPT= 359 and ICC=364/ ICC with OPT=385.

observation: no difference between the optimized and none optimized versions for GCC and ICC?

any thoughts? any suggestions for GCC or ICC default or optimized flags?  Thanks.

GCC with opt:

###########################
# Optimization flags      #
###########################
int=default=default=default:
notes0080=   Baseline C: -O3 -march=core2 -mfpmath=sse -funroll-loops -fasynchronous-unwind-tables
COPTIMIZE       = -O3 -march=core2 -mfpmath=sse -funroll-loops -fasynchronous-unwind-tables
notes0081=   Baseline C++: -O3 -march=core2 -mfpmath=sse -funroll-loops -fasynchronous-unwind-tables
CXXOPTIMIZE     = -O3 -march=core2 -mfpmath=sse -funroll-loops -fasynchronous-unwind-tables

ICC with opt:

#OPTIMIZATION
int=default=default=default:
notes0080= C: -xAVX -ipo -O3 -no-prec-div -opt-prefetch -opt-mem-layout-trans=3
COPTIMIZE=    -xAVX -ipo -O3 -no-prec-div -opt-prefetch -opt-mem-layout-trans=3
notes0081= C++: -xAVX -ipo -O3 -no-prec-div -opt-prefetch -opt-mem-layout-trans=3
CXXOPTIMIZE=  -xAVX -ipo -O3 -no-prec-div -opt-prefetch -opt-mem-layout-trans=3

[bouache]

Hi,

Trying to install SPEC2006. following ./install.sh I have this:

benchspec/CPU2006/998.specrand/data/ref/input/control: FAILED open or read
benchspec/CPU2006/998.specrand/data/ref/output/rand.234923.out: FAILED open or read
benchspec/CPU2006/998.specrand/data/ref/reftime: FAILED open or read
benchspec/CPU2006/998.specrand/data/test/input/control: FAILED open or read
benchspec/CPU2006/998.specrand/data/test/output/rand.24239.out: FAILED open or read
benchspec/CPU2006/998.specrand/data/test/reftime: FAILED open or read
benchspec/CPU2006/998.specrand/data/train/input/control: FAILED open or read
benchspec/CPU2006/998.specrand/data/train/output/rand.3.out: FAILED open or read
benchspec/CPU2006/998.specrand/data/train/reftime: FAILED open or read
benchspec/CPU2006/998.specrand/src/Makefile: FAILED open or read
benchspec/CPU2006/998.specrand/src/main.c: FAILED open or read
benchspec/CPU2006/998.specrand/src/specrand.c: FAILED open or read
benchspec/CPU2006/998.specrand/src/specrand.h: FAILED open or read

......

Is that normal???

Thanks

[KitturGanesh]

Hi MrB,
Just back from my vacation and saw your question. Well, it's been sometime ago that I ran spec but you shouldn't be getting such errors and I quickly checked on the Errata for Spec and it doesn't show such errors you mention so something else must be going on. I'll need to investigate this and may be check with our dev team who run spec bmarks with our compiler and will let you know, appreciate much.
Thanks,
Kittur

[KitturGanesh]

Hi MrB,
I chatted with Jin Lin my peer and was told that you now have no issue with Spec install. Let me know otherwise, appreciate much.
Regards,
Kittur

[bouache]

Thanks Kittur,

Yes I am working with Jin. he is helping me :)

[KitturGanesh]

Great, I'll sync up with Jin MrB, thanks.

[KitturGanesh]

Hi Mourad,
    I'll consider this issue resolved as we sync-ed up and all's taken care of, thanks!
Regards,
Kittur