ICC benchmarking and ICC vs GCC

[bouache]

Hi,

as you see I am new here in this forum.

I have a project comparing ICC vs GCC. I am looking for the best way to start this project. any links/ tutorial/ documentations are more than welcome.

thinking to do some benchmarking with both compilers using SPECInt or any other type of workloads/ programs.

Thanks for your help.

-Lx 

[KitturGanesh]

Hi Mourad,
Well, the best place to start is to skim through the user manual documentation on Optimization/features sections. There you'll find information on several optimization techniques from auto vectorization, parallelization, processor specific switches, IPO (inter procedural optimization) HPO (High perf optimization) etc. Of course, outside of this if you need to code etc., you'll find several parallelization models/techniques that you will be aware of (like OpenMP, CilkPlus (Array Notations, SIMD etc).

For Spec benchmarking you can look at AVX, AVX2, IPO, O3, -no-prec-div, -parallel -opt-prefetch etc...as a first shot and you can probe further after you read through the doc on optimization per-se.

Finally, there are several articles on optimization per-se that you can find in the Intel Developer Zone at  http://software.intel.com/ and go to the content for compilers and search for relavant knowledge base articles on performance guides etc under Development/Tools/Resources/Content Library section

Hope the above helps
_Kittur

[TimP]

Mourad B. wrote:

I have a project comparing ICC vs GCC. I am looking for the best way to start this project. any links/ tutorial/ documentations are more than welcome.

thinking to do some benchmarking with both compilers using SPECInt or any other type of workloads/ programs.

As you're probably aware, gcc doesn't have a group of optimizations to maximize SPECint performance under the benchmark rules.  Among your first steps are to determine, according to the manuals of both compilers, how consistent a set of options you wish to pick for each.

Assuming you are interested in benchmarking with a current architecture setting such as avx2, the set of options for gcc including -march=core-avx2 -ffast-math -O2 -ftree-vectorize -fno-cx-limited-range -fopenmp -funroll-loops --param max-unroll-times=2 -std=c99 (and several more) is rather complicated for consistency with icc -openmp -unroll2 -mavx2 -ansi-alias -std=c99.

In my experience, the avx2 support is good only in the latest icc release and gcc development version.  So there's another non-obvious decision which makes a big difference to your results.

Yet another are the particular (sometimes peculiar) choices of syntax each compiler prefers for optimization.

[KitturGanesh]

Good point Tim! Not all options that you can use with icc may be supported in gcc and your suggestions are certainly to the point....

[bouache]

Thanks a lot for your answers.

I will start testing, will let you know for the next step or any other issues. really appreciated 

--Lx 

[KitturGanesh]

Thanks, please let us know what you find and I'll pass it on to our product team - appreciate much.

[bouache]

I was trying to connect ICC to eclipse through these steps:

any help?

Thanks

1. Start Eclipse*.

2. Select Help > Install New Software.

3. Next to the Work with field, click the Add button. The Add Repository dialog opens.

4. Click the Local button and browse to the install_dir/eclipse_support/cdt8.0/eclipse directory.

5. Click OK.

6. Make sure Group items by category is not checked. (its not) I cant see any item. I installed ICC and it's in my /opt/intel/studioxx/composerxxx/

[KitturGanesh]

Hi MrB,
Pardon the delay, just saw your question :-( Well, you should follow the steps outlined in the product release notes under the section: "Eclipse Integration" and you should be able to do so.  You'll need a proper version of Eclipse, CDT and a functional JRE installed and configured in your environment before you try and install. Under the section there in the release notes, you'll notice a section "How to Install the Intel C++ Eclipse Product Extension in Your Eclipse Platform" which you can follow...

_Regards,
Kittur

[bouache]

HI Kittur,

Thanks for your reply.

I did check all requirements and everything looks normal.

• Eclipse 3.7, 3.8 or 4.2
  [http://www.eclipse.org/downloads/]
• CDT 8.0 or later
  [http://www.eclipse.org/cdt/]
• Java Runtime Environment (JRE) version 6.0 (also called 1.6) update 11 or later
  [http://www.oracle.com/technetwork/java/javase/downloads/index.html]
• Intel® Composer XE 2013 (separate or any suite that provides it, like Intel® Parallel Studio XE 2013)
  [http://software.intel.com/en-us/intel-composer-xe]

so following "Click on the Local… button, specify the directory containing the Intel C++ Eclipse extension and confirm." I don't find any item?!

is there an other way to do it?

Thanks,

[KitturGanesh]

Hi MrB,

Yes you can just install the Composer XE if you just want to use the compiler per-se. BTW, make sure you have the compiler installed and the dir below exists. For example if you have installed the latest composer XE as root, you'll find the directory with the the Intel C++ Eclipse Product extension as below:
     cd /opt/intel/composer_xe_2013.<n>.<xxx>/eclipse_support/cdt8.0/eclipse

So to add the Intel C++ product extension to your existing Eclipse configuration, follow these steps, from within Eclipse.

1) Open the "Available Software" page by selecting: Help > Install New Software...Click on the "Add..." button. Select "Local...".

A directory browser will open. Browse to select the eclipse directory in your Intel C++ compiler installation. For example, if you installed the compiler as root to the default directory, you would browse to

    /opt/intel/composer_xe_2013.<n>.<xxx>/eclipse_support/cdt8.0/eclipse

Select “OK” to close the directory browser. Then select "OK" to close the “Add Site” dialog.
Select the two boxes for the Intel C++ integration:  there will be one box for “Intel® C++ Compiler Documentation” and a second box for “Intel® C++ Compiler XE 14.0 for Linux* OS”.

NOTE:
  *  The Intel features will not be visible if you have Group items by category set – unset this option to view the Intel features.
  *  Click the “Next” button. An “Install” dialog will open which gives you a chance to review and confirm you want to install the checked items. Click “Next”. You will now be asked to accept the license agreement. Accept the license agreement and click “Finish”. Select “OK” on the “Security Warning” dialog that says you are installing software that contains unsigned content.
The installation of the Intel support will proceed.

  * When asked to restart Eclipse, select “Yes”.
  * When Eclipse restarts, you will be able to create and work with CDT projects that use the Intel C++ compiler.

Also note: If your installation of Eclipse already has an earlier Intel® C++ Composer XE integration
installed, installing the updated integration will not work. You will need to install a fresh version
of Eclipse into which you can install the latest Composer XE integration. For this same reason,
using the Eclipse update mechanism to install a newer Composer XE integration will not work.
=====================================================================

Let me know if the above helps?

_Kittur

[bouache]

Thanks for your reply.

Here what I have when I try to add the ICC features to my eclipse.

Thanks

[KitturGanesh]

Hi, I see that the "Group items by Category" is enabled in the snapshot you show. Disable or uncheck that item. Once you do, the Intel features should become visible.... _Kittur

[bouache]

done

Thanks. I am trying to foloow the tutorial.

In /opt/intel/composer_xe_2013_sp1.2.144/Samples/en_US/C++/cilk_samples, I am trying to write a Makefile to compare the gcc vs icpc performance in qsort-mutex, qsort, matrix, linear-recurrence, matrix-multiply.

any idea or links for that type of comparison. The idea is just check with the same options if icpc is better than gcc and why?

Thanks

[bouache]

an other question: How can I use the vectorization option with GCC to compare with the vectorized version of ICC. Here what I mean:

/opt/intel/composer_xe_2013_sp1.2.144/Samples/en_US/C++/vec_samples

trying with gcc for the default option:

g++ Driver.o Multiply.o -o default

 ./default

ROW:101 COL: 101
Execution time is 30.162 seconds
GigaFlops = 0.676403
Sum of result = 195853.999899

trying with icc for the default option:

icc Driver.o Multiply.o -o default

./default

ROW:101 COL: 101
Execution time is 30.197 seconds
GigaFlops = 0.675631
Sum of result = 195853.999899

ICC after vectorization:

# ./default

ROW:101 COL: 101
Execution time is 8.740 seconds
GigaFlops = 2.334442
Sum of result = 195853.999899

Thanks

[TimP]

While there is a cilkplus branch of gcc, and some of that is merged into gcc 4.10, gcc cilkplus isn't ready for release.   Nor does icc Cilk(tm) Plus offer competitive performance on other than a few selected applications.

I think you have missed the fact that g++ defaults to -O0.  Boosters of one compiler or another hope that you don't study the options of the others, but this is out of bounds.  Here are some roughly equivalent option groups:

icpc -unroll2 -xHost -ansi-alias

g++ -O2 -ftree-vectorize -ffast-math -funroll-loops --param max-unroll-times=2 -march=native -fno-cx-limited-range

or

g++ -O3 -ffast-math -funroll-loops --param max-unroll-times=2 -march=native -fno-cx-limited-range

If you use complex math, you should study the g++ -fcx-limited-range and equivalent icpc -complex-limited-range options.

I don't recall how the icc vector samples suggest dealing with aliasing.  The semi-portable way, which works with both icpc and g++, is with the * __restrict qualifiers.  Only the latest development versions of g++ support #pragma GCC ivdep.

For matrix algebra, icpc offers MKL library function options, or -O3 may be useful.  OpenMP programming (inherent in MKL and ACML) will be important for both icpc and g++. 

Although standard linux distros offer lapack and blas libraries compatible with g++, they tend to be compiled without full optimization or a current version of gfortran, and without OpenMP.  Due to the importance of threaded vector libraries such as MKL and ACML, matrix algebra isn't necessarily a meaningful arena for comparing compiler performance.

[KitturGanesh]

Tim has eloquently given you the correct info on Cilk etc. Like Tim mentioned, on vector samples use the __restrict qualifier to ensure non ambiguity and compiler can go ahead and vectorize...... _Kittur

[TimP]

I did mistakenly omit icpc option -ansi-alias   to assert that your source code complies with the rules about typed aliasing.  The equivalent g++ option -fstrict-aliasing is set by default.

Which reminds me I must check whether the icpc beta test compiler has changed this default.

[KitturGanesh]

Tim, I did check 15.0 beta and ansi-alias is enabled by default and to disable you'll have to use -no-ansi-alias option explicitly.... _Kittur

[TimP]

-ansi-alias apparently is intended to be default for linux going forward, but not for Windows. It's not consistently documented. It seems there will be now more cases where the option makes no difference, possibly due both to better optimization in some cases and to the compiler exerting more caution in other cases.

[bouache]

thanks for all your replies.

I am trying to build a config file for SPEC 2006 using ICC with the same flags as in GCC, like:

#
# NOTE: The F90 benchmarks will *not* work with this compiler
#       setting.
default=default=default=default:
CC      = icc
CXX     = icpc
FC      = ifort
F77     = g77

I know that FC is ifort, what about F77 (its a GCC config)?

if you have a ready SPEC2000 or 2006 for ICC, it will be very helpful.

I have also an other issue running SPEC2000, see here:

                 *MatrixId, MatrixTkn.DbId, MatrixTkn.Handle))
                                            ^

oa1.c(1714): error: case label value has already appeared in this switch at line 1705
      case sizeof (tokentype )  :
           ^

oa1.c(1851): error: case label value has already appeared in this switch at line 1842
      case sizeof (tokentype )  :
           ^

compilation aborted for oa1.c (code 2)
specmake: *** [oa1.o] Error 2
specmake  options 2> options.err | tee options.out
COMP: icc -c -o options.o              -O3 -funroll-all-loops
LINK: icc       -O3 -funroll-all-loops     -lm  -o options
  Some files did not appear to be built: vortex
*** Error building 255.vortex
If you wish to ignore this error, please use '-I' or ignore errors.

Thanks a lot for your help.

---mb

[Steven_L_Intel1]

g77 is an obsolete Fortran 77 compiler that was part of gcc. I would hope nobody is using it for SPEC anymore.