Can't profile my threaded Application

[r4dium]

I have programmed a Win32 Application based upon DirectX9 and a video library called
FFMpeg with Visual Studio Team System 2008 under Windows Vista Business using MS C++ 9.0 .

When I try to profile the executable by Thread Profiler the following error message pups up:
"TP file not found. The target executable may have exited unexpectedly."
This correlates with the message vista gives me shortly before, which states that the program doesn't work any more.

It seems that something prevents my executable to start. If I simply double klick on it, it starts perfectly.

I am using the /Zi, /fixed:no and /MTd switches.

Regards R4DIUM

[Mark_D_Intel]

Are there any messages in the output windows that might give more information (like failed instrumentation or something)?

For standalone VTune, the output window has a drop-down box for General and Instrumentation Results - particularly check the Instrumentation Results output.
If integrated into VS, all the output should be together in the 'VTune Performance Tools' window.

Mark

[r4dium]

I am using stand-alone VTune.
There are no errors in the instrumentation output, just a warning that the
msvcr90.dll and a few other msv*90.dll are certified and that the instrumentation is not applicable for it.
All other dlls show a success, but after "going to a deeper instrumentation level".

The General-Output contains nothing.

If I create a MFC application I can profile it without problems - so I think it is something project specific.

VTune V 9.0 Build 719
Thread Profiler 3.1 Build 0.26185
running under Windows Vista 32-bit SP1

Regards

R4DIUM

[Mark_D_Intel]

Try upgrading Thread Profiler to update 10.

If that doesn't work, try lowering the default instrumentation level, and raising it back for the modules you are interested in.

Mark

[r4dium]

I upgraded, but the problem is still there.


However, I localized the problem....

Whenever I use some function of "avcodec-51.dll" Thread Profiler is unable to start the application correctly.
Moreover, it is unenteresting if the function is executed during runtime - to cause the problem it is enough
to use such a function in a never activated class of your program [not created and reached from main()].

Lowering instrumentation on "avcodec-51.dll" by switching to "Module Imports" did also not help.

A few informations concerning "avcodec-51.dll":

1.) It is a C library - So I use:extern "C"{ include }
2.) It is a Linux library originally, most likely compiled with MinGW under some Windows (probably not Vista)

The developer of avcodec-51.dll says:
"You will not be able to debug inside the libav* libraries, since MSVC++ does not recognize the debug symbols generated by GCC."

Some more ideas how I could procceed or what could cause the problems?


Regards

R4DIUM

[Mark_D_Intel]

At first I thought that Thread Profiler would not handle GCC-compiled code on Windows, but I installed MinGW and it worked on some simple examples. Thread Profiler also worked on ffmpeg.exe from sourceforge.

It appears there is something particular about this library causing a problem.

I have not had success in building libavcodec. Could you get me a copy of the library somehow ? One of the following should work
- directions on how to build it (in particular, what are the correct versions for msys and supporting tools)
- point to a download location
- attach the dll,lib, and include files here
- upload them to ftp.intel.com/pub/incoming (and tell me the exact name- for security reasons the directory listing cannot be seen)

Mark

[r4dium]

libavcodec-51.dll is part of the FFMPEG project at:
http://ffmpeg.mplayerhq.hu

To build in under Windows you have to follow these steps:
http://ffmpeg.mplayerhq.hu/general.html#TOC10

You can either download it subversioned at
http://ffmpeg.mplayerhq.hu/ffmpeg-checkout-snapshot.tar.bz2
or only bare sources at:
http://ffmpeg.mplayerhq.hu/ffmpeg-export-snapshot.tar.bz2

I uploaded the FFMPEG files I use to the incoming directory , the file
is named "FFmpegR4DIUM.zip"

As I already wrote I didn't compile it by myself, but instead downloaded it
compiled by someone else from a download location reachable through:
http://arrozcru.no-ip.org/ffmpeg/
Unfortunately this website is not reachable at the moment.

Furthermore I must warn you: Compilation of FFMPEG under windows
can be very time consuming. I did it once and it was a nightmare.
Maybe its better to start with the lib/dll files I use and then
debug it as a first approach.

Thanks very much for all the efforts you put in this matter!

Regards

R4DIUM

[r4dium]

Are there any new insights into the libavcodec.dll issue? Can I help somehow, maybe by trying to gather new
informations of some kind.

Regards

R4DIUM

[Mark_D_Intel]

I was able to build avcodec.dll and can reproduce the issue. Still looking into it.

Mark

[r4dium]

Thx for you efforts so far,

can you give me a time estimate how long you think the problem will be pending?


Regards

R4DIUM

[Mark_D_Intel]

There may be a work-around, but it involves building libavcodec.

The steps are
1. Follow the instructions and run configure and make
2. Remove *.dll, *.lib, and *.a files from the libavcodec directory
3. Run 'make' again. Then do the 'make install' step.

This creates a library that works with Thread Profiler.

I don't know how long it will take to find or fix the actual problem.


Mark

[r4dium]

I will follow your proposal.
If the lib works after re-compilation, there is no need for you
to invest more of your spare time into this issue.

Best Regards

R4DIUM

[r4dium]

I followed your proposal, but without success, thread profiler still exits
unexpectedly.

I used this guide to compile ffmpeg:
http://ffmpeg.arrozcru.com/ffmpeg_wiki/tiki-index.php

Moreover I used the newest gcc version available for MinGW:
http://ffmpeg.arrozcru.com/ffmpeg_wiki/tiki-index.php?page=Installing+MinGW
According to the guide only the newest gcc version is capable of producing working dlls.

I will try to develop my software with statical ffmpeg-libs if you don't have some advice.

Regards

R4DIUM

[Mark_D_Intel]

I suspect the problem lies with the import table of avcodec.dll, which contains multiple entries for avutil.dll. After re-linking (with the work-around I described), the import table contains no entries for avutil.dll.

If the import table is the problem, then static linking would also be a good work-around.

Mark

[r4dium]

Static linking did the job for me!

Seems you are right with your assumption about the import tables.


Thx a lot

R4DIUM

[bsantana]

I'm having same issue "TP file not found. The target .....". I was looking these posts but I can't understand how to fix this issue =( could you help me please?

I have 5 systems with Windows Vista and Intel Thread Profiler, and on all my systems I get the same error message, I was thinking that this problem was caused for some Windows patch update (not quite sure).

Thanks in advance

Benjamin Santana

[r4dium]

Quoting - bsantana

I'm having same issue "TP file not found. The target .....". I was looking these posts but I can't understand how to fix this issue =( could you help me please?

I have 5 systems with Windows Vista and Intel Thread Profiler, and on all my systems I get the same error message, I was thinking that this problem was caused for some Windows patch update (not quite sure).

Thanks in advance

Benjamin Santana

Investigate further by:

1.) Terminate the program gracefully by pressing it's window's close button or another built in shutdown mechanism,

but not by pressing VTunes red stop button during the profile run.

2.) If this does not help go to Menu->Configure->Modyfiy->Instrumentation->Advanced and set everything to "Module Imports" except the executable. Does it work in this case ( also procceed like in 1. )? If yes, investigate further by incrementally resetting the libs to API Imports. One single lib could cause the problem (had this problem with DirectX9 Lib).

3.) Do an instrumentation during compilation by using the /Qtprofile switch in the compiler command line. A *.tp

file will be genrated after running the executable which you can open with Thread Profiler. Does it work in this case?

4.) If nothing helped, built a statically linked executable - does it work in this case?

Regards R4DIUM

[bsantana]

Thanks for your time on reply my post R4dium.

I was working on your suggestions with the following results: 1) and 2) didnt work, I continue getting same error message.

Option 3) generate a tp.tp file after I run it, good news is that I can open this file with Thread Profiler, bad news is that results not make sense to me, I'm going to try explain better. The program that i want to profile was parallelized with OpenMP. I may check it with Thread Profiler with OpenMP -specific collection mode fine, in this view I can see thread times fine and actually most of the time the program runs with parallel regions. Then I want to verify my program with Instrumentation collection mode, here is where I have the issue. After I worked with option 3) you suggested me, I may open it the file, but the report tells me that I have my threads the whole time in wait mode and show me that only one thread perform whole the job in serial mode, I mean, same program give two different results: one with OpenMP specific collection mode and different result with Instrumentation collection mode by getting profile data with /Qtprofile switch. So, at this point I don't know if profiling with /Qtprofile switch is reliable. I'm wondering if you have results with this method that make sense to you ?

And to be honest I do not know howto perform option 4) you suggested me, I'm sorry =(

I'm going to continue investigating and if I found something new, I'll post news here.

Best Regards

Benjamin Santana

[r4dium]

Try to further isolate the problem:

If you only use a few libs, it might be useful to find out

if profiling works if you comment out a specific lib's function calls.

It might be more comfortable if you program a test-main() routine for each lib

were you only call the functions of a certain lib and look if this program is profilable

in thread profiler.

This is the only way to make sure that no lib is responsible for the unexpected exit.

In my case it was a lib called "libavcodec.dll" which caused the same error message

even if the functions where never calles during runtime (was enough to compile in such functions).

Regards

R4DIUM

[Mark_D_Intel]

Quoting - bsantana

Thanks for your time on reply my post R4dium.

I was working on your suggestions with the following results: 1) and 2) didnt work, I continue getting same error message.

Option 3) generate a tp.tp file after I run it, good news is that I can open this file with Thread Profiler, bad news is that results not make sense to me, I'm going to try explain better. The program that i want to profile was parallelized with OpenMP. I may check it with Thread Profiler with OpenMP -specific collection mode fine, in this view I can see thread times fine and actually most of the time the program runs with parallel regions. Then I want to verify my program with Instrumentation collection mode, here is where I have the issue. After I worked with option 3) you suggested me, I may open it the file, but the report tells me that I have my threads the whole time in wait mode and show me that only one thread perform whole the job in serial mode, I mean, same program give two different results: one with OpenMP specific collection mode and different result with Instrumentation collection mode by getting profile data with /Qtprofile switch. So, at this point I don't know if profiling with /Qtprofile switch is reliable. I'm wondering if you have results with this method that make sense to you ?

And to be honest I do not know howto perform option 4) you suggested me, I'm sorry =(

I'm going to continue investigating and if I found something new, I'll post news here.

Best Regards

Benjamin Santana

With regards to option 3, the compiler-instrumentation option (/Qtprofile) doesn't work well with OpenMP because all the threading calls in the Openmp library won't get instrumented. You will need to use Openmp-specific profile information (/Qopenmp_profile), or get binary instrumentation working.

[robert_c_3]

Quoting - r4dium

I have programmed a Win32 Application based upon DirectX9 and a video library called
FFMpeg with Visual Studio Team System 2008 under Windows Vista Business using MS C++ 9.0 .

When I try to profile the executable by Thread Profiler the following error message pups up:
"TP file not found. The target executable may have exited unexpectedly."
This correlates with the message vista gives me shortly before, which states that the program doesn't work any more.

It seems that something prevents my executable to start. If I simply double klick on it, it starts perfectly.

I am using the /Zi, /fixed:no and /MTd switches.

Regards R4DIUM

Just one more data point - I am also getting this error on my home system - quad core system running Vista

I am usingIntel VTune Performance Analyzer 9.0

Build:719

Get the same error just trying to profile the primes example that comes in the Vtune TProfiler Smpales folder - This is much simpler reproducer if you can get it to work on Intel system on Vista as it only requiress the following dlls

Kernel32.dll

msvcr90d.dll

ntdll.dll

primesbalanced.exe

I have tried running VTune as administrator etc no luck

It just does not seem to create a tp file at all

The TP cahce folder is set to default here:

C:UsersbobAppDataLocalTempCacheTP_Cache

running Thread Profiler places several temp files here but NO TP file is created - I wonder if it is an over zealous Vista protection issue of some kind

Thoughts?

bobc