topic Re: The Thread Profiler OpenMP*-specific" collector in Analyzers

The Thread Profiler OpenMP*-specific" collector

jansson — Tue, 01 Dec 2009 12:24:49 GMT

Hi,

I have problem getting the "Intel Thread Profiler OpenMP*-specific" collector to work in Thread Profiler. I tried Stand alone, in Thread Profiler, in Visual Studio 2005.
I use it to get my Fortran program to scale well, it uses OpenMP 3.0 and therefore I have to use the intel fortran compiler 11.1 (I use 11.1.051).

I have installed:
-------------------------------------------
Intel VTune Performance Analyzer 9.1
Build:210

Intel Thread Profiler 3.1
Build:0.27583

Intel Thread Checker 3.1
Build:0.27583

Visual Studio 2005

Intel Visual Fortran Compiler Integration for Microsoft Visual Studio* 2005, 11.1.3468.2005

Intel Visual Fortran Compiler Professional 11.1.051

Windows XP SP3, Core 2 Duo
-------------------------------------------

I run VS2005 and Vtune et.al. from my normal user account.
If I as Administrator run
C:\Program Files\Intel\VTune\Analyzer\bin\elevateUserRight.exe account verify
for my user account (Debug user), it report...
-------------------------------------------
Elevate user rights utility.
Copyright (C) 2006 Intel Corporation. All rights reserved.

"Counter Monitor" is available
"Sampling" is available
-------------------------------------------

In my Visual studio solution I got many Fortran projects, to each I added the
Fortran - command line - additional options
/Qopenmp_profile /MD /Qopenmp-lib:compat /Qopenmp-link:dynamic

Here is the compiler settings for my main project
Fortran - command line - all options:
-------------------------------------------
/nologo /debug:minimal /fpp /I"..\..\src\incl32" /I"..\..\src\incl" /I"..\src\main" /I"..\..\src\diglib\incl" /I"..\..\dictra\src" /I"..\..\win32ivf\diglib\release" /DWIN32 /D_VF_VERSION_=660 /D_THERMOCALC /iface:cvf /module:"Release OpenMP Thread Profiler\" /object:"Release OpenMP Thread Profiler\" /check:pointer /check:bounds /check:format /check:output_conversion /check:arg_temp_created /libs:dll /threads /c
-------------------------------------------

..the main project Fortran - Linker - additional options
/fixed:no /LTCG

Here is the Linker settings for my main project
Linker - command line - all options:
-------------------------------------------
/OUT:"Release OpenMP Thread Profiler\TCC.exe" /INCREMENTAL:NO /NOLOGO /NODEFAULTLIB:"msvcrt.lib" /MANIFEST /MANIFESTFILE:"D:\GibbsOpenMP\Win32ivfc\Win32ivfc\Release OpenMP Thread Profiler\TCC.exe.intermediate.manifest" /DEBUG /PDB:"D:\GibbsOpenMP\Win32ivfc\Win32ivfc\Release OpenMP Thread Profiler\TCC.pdb" /SUBSYSTEM:CONSOLE /STACK:300000000 /ENTRY:"WinMainCRTStartup" /IMPLIB:"D:\GibbsOpenMP\Win32ivfc\Win32ivfc\Release OpenMP Thread Profiler\TCC.lib"
-------------------------------------------

There is also a C project with no "additional options", using the C-compiler in Visual Studio 2005 Standard Edition. No OpenMP directives are used in the c-files.
C/C++ -command line- all options:
-------------------------------------------
/O2 /GL /D "WIN32" /D "_UNICODE" /D "UNICODE" /FD /EHsc /MD /openmp /Fo"Release OpenMP Thread Profiler\" /Fd"Release OpenMP Thread Profiler\vc80.pdb" /W3 /nologo /c /Wp64 /Zi /TC /errorReport:prompt
-------------------------------------------

In the "serial" version of the code there is 3 threads. One is running the actual programme, one is processing the messages from windows and sending them to the console, and one is checking the license for our software.

Below are the result from my failed attempts when I tried OpenMP specific profiling
1) Stand alone
2) Thread Profiler
3) Visual Studio 2005

1)
If I run the exe file directly it seems to run fine, I get the guide.gvs file after its done, but its quite brief.
start guide.gvs --------------------------------------------

*** KAI statistics library k3301

*** Begin Task 0

Environment variables:
OMP_NUM_THREADS : 1
OMP_SCHEDULE : static
OMP_DYNAMIC : FALSE
OMP_NESTED : FALSE
KMP_STATSFILE : guide.gvs
KMP_STATSCOLS : 80
KMP_INTERVAL : 0
KMP_BLOCKTIME : 200
KMP_PARALLEL : 3
KMP_STACKSIZE : 2097152
KMP_STACKOFFSET : 0
KMP_SCHEDULING :
KMP_CHUNK :
KMP_LIBRARY : throughput
end

System parameters:
start : Tue Dec 01 11:19:54 2009
stop : Tue Dec 01 11:20:13 2009
host : FLIN
ncpu : 2
end

Region counts:
serial regions : 1
barrier regions : 0
parallel regions : 0
end

Program execution time (in seconds):
cpu : 9.38 sec
elapsed : 19.22 sec
serial : 19.22 sec
parallel : 0.00 sec
cpu percent : 48.79 %
end

Summary over all regions (has 1 threads):
# Thread #0
end

Serial program regions:
Serial region #1 executes for 19.22 seconds
begins at START OF PROGRAM
ends at END OF PROGRAM
end

Serial region #1 profile:
# Thread Incl Excl Routine
end

Program events (total):
# Thread #0
mppbeg : 1
mppend : 1
serial : 1
end

Serial section events:
# Serial #1
mppbeg : 1
mppend : 1
serial : 1
end

*** end
end guide.gvs --------------------------------------------

2)
If I make a project in Thread Profiler and run from there I get first:
--------------------------------------------
"None of the intel Thread profiler OpenMP* collector's modules of interest have been linked to an OpenMP library.

This collector may not generate any result.

Do you wish to continue?"
--------------------------------------------

Pressing OK I see my program start and then I get.
--------------------------------------------
OMP: Warning #2: Cannot open message catalog "1053\libiomp5ui.dll":
OMP: System error #126: The specified module could not be found.
OMP: Info #3: Default messages are used.
OMP: Error #136: Cannot create thread.
OMP: System error #8: Not enough storage is available to process this command.
--------------------------------------------

Then my program terminates by it self without delivering any results, and I get a new pop up window informing me of..
--------------------------------------------
Note that partial results cannot be collected when Intel thread profiler for OpenMP* activities are stopped. Try ending the application from within the application itself. Alternatively select Configure > Modify Collector and set Dump Statistics to periodically save results.
--------------------------------------------

I click OK and get a new pop up.
--------------------------------------------
No results were generated for this run. Check the log or output window for further information.
--------------------------------------------
I click OK, and no data is visible for the run.

3)
If I do this in VS2005 I get the same pop up as when using Thread Profiler, and I get the following output that looks fine.
--------------------------------------------
Tue Dec 01 12:08:30 2009 Preparing to collect parallel performance data...
Tue Dec 01 12:08:34 2009 Collecting parallel performance data...
Tue Dec 01 12:08:57 2009 Completed collecting parallel performance data.
--------------------------------------------

Do you see any error in my set up preventing the OpenMP specific Thread Profiling version to run?
All comments are welcome!

The second question:
If I instead choose "Thread Profiler instrumentation" I am only able to instrument at the "API imports" level. But I would like to do it more fully, what may prevent this?

Cheers,
Magnus

Re: The Thread Profiler OpenMP*-specific" collector

TimP — Tue, 01 Dec 2009 13:30:06 GMT

It's hard to see the trees in your forest of text posting.
For generating guide.gvs, it should be sufficient to compile with /Qopenmp and link with /Qopenmp_profile. A release build is fine, the /Qopenmp_profile link simply turns on the generation of guide.gvs. Each parallel region compiled by Intel compilers with /Qopenmp which is executed would generate its own section in guide.gvs, named according to the source line of the omp parallel. It looks as if you didn't turn on /Openmp (which you would do by the specific Visual Studio setting).
OpenMP parallel regions in MSVC don't get recognized with specific sections in guide.gvs, but they figure in the overall statistics.
Thread Profiler should work as long as you built normally with /Qopenmp, regardless of whether link is done with /Qopenmp (no guide.gvs generated) or /Qopenmp_profile. If you have Windows thread calls in addition to OpenMP, you would likely want /Zi symbols so as to help Thread Profiler give you more detail.

Re: The Thread Profiler OpenMP*-specific" collector

jansson — Tue, 01 Dec 2009 15:53:02 GMT

Thank you Tim18 for reading and answering my massive posting!

Quoting - tim18

I wanted to include all important info at once, but then I find much info inportant after trying to get this working for a few days and reading many Knowledge Base articles.

Do I understand you correctly when I in VS2005 for all Fortran Projects modify (compile) properties under "Fortran" - "Command line" - removing/Qopenmp_profile and adding /Qopenmp. Then in "property pages" - "Linker" - "Command line" I add /Qopenmp_profile.
Probably not because I get 2>Link: warning #10006: ignoring unknown option '/Qopenmp_profile' when doing soo.
What is my mistake?

This might be because of the many trees in my forest. I believe I turn on OpenMP compilation by adding to
"Fortran" - "command line" - "additional options" the /Qopenmp_profile switch
According to Preparing Your Software I must not use both /Qopenmp and /Qopenmp_profile, if I do anyway it say it ignores /Qopenmp.

Im not familiar with the abreviation MSVC. I guess it means Microsoft Visual C/C++ or its compiler/runtime. But there is no OpenMP pragmas in my C code only in my Fortran code. I am a pure Fortran programer and edit C mostly by guessing.

For the compile, the C code have /Zi for debug and the Fortan code have Debugg using "Line Numbers Only" (/debug:minimal). For linking its also on using (/DEBUG).

Cheers,
Magnus

Re: The Thread Profiler OpenMP*-specific" collector

jansson — Wed, 02 Dec 2009 16:35:06 GMT

I really miss a Fortran example collection for Intel Thread Profiler to get started so I made a basic example using the OpenMP TASK construct I want to profile in my own code.

My first problem is that I cant get higher than API instrumentation using Thread Profiler, am I wrong expecting this?
The second problem is I cant get OpenMP specific profiling to work at all.

In the attachment FortranOpenMP.zip I includes the full project except binaries and data to keep file slim.
I only added /DEBUG, multi threaded librarys and /Qopenmp compared to the default project.
I use Visual Studio 2005 and ifort 11.1.051.

Please test if you can reproduce my problem, I would appreciate indication of both success and failure to narrow down where I go wrong using Intel Thread Profiler.

Thank you,
Magnus

Re: The Thread Profiler OpenMP*-specific" collector

TimP — Wed, 02 Dec 2009 21:35:36 GMT

couldn't quickly fix the VS project.
ifort /Qopenmp-profile ../*.f90 *.f90
running on dual core produces guide.gvs:
....
Program execution time (in seconds):
cpu : 7.42 sec
elapsed : 8.67 sec
serial : 3.57 sec
parallel : 5.10 sec
cpu percent : 85.64 %
end

Summary over all regions (has 4 threads):
# Thread #0 #1 #2 #3
Sum Parallel : 5.097 5.061 5.088 5.061
Sum Barrier : 2.759 5.052 5.087 5.052
Sum Imbalance : 0.009 0.009 7.0e-005 0.009
Min Parallel : 0.009 0.009 3.8e-004 0.009
Min Barrier : 2.759 5.052 5.087 5.052
Min Imbalance : 0.009 0.009 7.0e-005 0.009
Max Parallel : 5.088 5.052 5.087 5.052
Max Barrier : 2.759 5.052 5.087 5.052
Max Imbalance : 0.009 0.009 7.0e-005 0.009
end

Region #1 (has 4 threads) at _MAIN__/30 in "C:Documents and SettingstcprinceM
y DocumentstimtimsrcnetmagnusFortranOpenMPparallelparallel.f90" (has 1 b
arriers):
# Thread #0 #1 #2 #3
Sum Parallel : 5.088 5.052 5.087 5.052
Sum Barrier : 2.759 5.052 5.087 5.052
Min Parallel : 5.088 5.052 5.087 5.052
Min Barrier : 2.759 5.052 5.087 5.052
Max Parallel : 5.088 5.052 5.087 5.052
Max Barrier : 2.759 5.052 5.087 5.052
end

Re: The Thread Profiler OpenMP*-specific" collector

jansson — Thu, 03 Dec 2009 14:17:06 GMT

Running the .exe file I get a guide.gvs using /Qopenmp and tp.tp using /Qtprofile. I can then open both in Thread Profiler to get the graphical view and it looks as expected.
But I still cant use these .exe file directly from within the Thread Profiler GUI to profile the exe file.

I will uninstall Intel Thread Profiler and Checker and install them again and report back if this helps me get things working as they should.

I now suspect that I got my problem because I previously installed and uninstalled Inspector & Amplifier (after instaling TP+TC). I uninstaled them bacause they somtimes crached Vtune for me and they are presently only for C/C++ as I understand it and I only do Fortran.

Cheers,
Magnus

Re: The Thread Profiler OpenMP*-specific" collector

jansson — Fri, 04 Dec 2009 17:16:32 GMT

I now reinstalled Vtune, Thread Checker, Thread Profiler, and ifort 11.1.51.
I checked that they are of todays latest versions.
The behaviour is as strange as before, it works outside TP not inside.

Any ideas of what to test next?

Do anyone have a VS2005 Fortran project to post here that they can profile in Thread Profiler using ifort 11.1.51 and including a OpenMP TASK construct?
You are welcome to use the source code from my project posted above if convenient.

Cheers,
Magnus

Re: The Thread Profiler OpenMP*-specific" collector

Mark_D_Intel — Wed, 30 Dec 2009 21:07:14 GMT

Thread Profiler in Instrumentation mode only needs API imports (to track the threading API's). Higher levels of instrumentation are used by Call Graph and Thread Checker.

Thread Profiler in OpenMP-specific mode doesn't work well with libiomp5.dll. The best workaround is to use /Qopenmp_profile and load the guide.gvs into the Thread Profiler GUI after the run (as you have been doing).
It should also work to use /Qopenmp_profile, run through Thread Profiler and ignore the warnings about the exe not being linked with an OpenMP runtime. Although this appears not to work for you (possibly a PATH issue or manifest issue with finding the OMP library ?)