Inspector XE hangs on ffmpeg program

[ywsong]

I just updated Intel Inspector XE to update 5 version.

One of program I was using hangs. It worked fine on previous version.

CPU: Intel(R) Core(TM)2 Duo CPU

OS: Ubuntu 12.04 with kernel 3.2.0-39.

Used ffmpeg-0.6.3 with input file atttached but it does not stop for more than 24 hours.

The exact command line was: (it also hangs on GUI version)

inspxe-cl -collect ti-3 -module-filter-mode=include -appdebug=off -app-working-dir (path of ffmpeg-0.6.3)  -- (ffmpeg binary) -threads 2 -i test_10.AVI -target ntsc-dvd -s 1280x720 out_10.MOV

[SergeyKostrov]

>>...Used ffmpeg-0.6.3 with input file atttached but it does not stop for more than 24 hours. Please provide more details for the ffmpeg-0.6.3 program. Is there a link to a download?

[ywsong]

ffmpeg is in:

http://www.ffmpeg.org/download.html

They don't have ffmpeg-0.6.3 anymore but you can try it using version 0.6.6.

I think version 0.6.6 has the same problem.

You can see it on the second entry from the bottom of the page.

[SergeyKostrov]

I see that you're using older version of ffmpeg.exe. Next week I'll be able to look at a newer version ( Release configuration ) of ffmpeg.exe: ... ffmpeg version N-50911-g9efcfbe Copyright (c) 2000-2013 the FFmpeg developers built on Mar 13 2013 21:26:48 with gcc 4.7.2 (GCC) ...

[Bernard]

@ywsong

Is there is a possibility to collect a process dump on Ubuntu?

[ywsong]

I dumped the core but the size is too big (over 400MB) and it does not allow to attach it.

[SergeyKostrov]

>>... >>OS: Ubuntu 12.04 with kernel 3.2.0-39. >> >>Used ffmpeg-0.6.3 with input file atttached but it does not stop for more than 24 hours. >>... Even if my test is not done yet I will do it on a 64-bit Windows 7 Professional and I hope it doesn't look like useless.

[Bernard]

ywsong wrote:

I dumped the core but the size is too big (over 400MB) and it does not allow to attach it.

Is there an option in GDB to perform an automatic analysis of crashed application?

[ywsong]

iliyapolak wrote:

Quote:

ywsongwrote:

I dumped the core but the size is too big (over 400MB) and it does not allow to attach it.

Is there an option in GDB to perform an automatic analysis of crashed application?

It does not crash but hangs.

I looked at it using linux top command and the process was not active.

[Bernard]

Sorry  of course it hangs.Is there any tool on Linux which can analyze hang process? Windows has AD plus.

In your earlier post you mentioned that  dump has 400mb of data.Does it contain also kenel mode part?

[SergeyKostrov]

I couldn't reproduce the problem on a Windows 7 Professional. One thing I've noticed in your command line for the utility is a -thread option. Try to use -thread 1 ( or don't use it at all ) instead of -thread 2.

[ywsong]

Sergey Kostrov wrote:

I couldn't reproduce the problem on a Windows 7 Professional. One thing I've noticed in your command line for the utility is a -thread option. Try to use -thread 1 ( or don't use it at all ) instead of -thread 2.

Thanks.

But Windows is a totally different platform as Ubuntu (Linux).

[ywsong]

iliyapolak wrote:

Sorry  of course it hangs.Is there any tool on Linux which can analyze hang process? Windows has AD plus.

In your earlier post you mentioned that  dump has 400mb of data.Does it contain also kenel mode part?

The following is from using strace tool in Linux,

~$ ps -a
  PID TTY          TIME CMD
 3810 pts/2    00:00:00 inspxe-cl
 3814 pts/2    00:00:00 inspxe-runtc
 3820 pts/2    00:00:07 ffmpeg_g
 3858 pts/1    00:00:00 ps

~$ strace -p 3810
Process 3810 attached - interrupt to quit
futex(0xbfda5e4c, FUTEX_WAIT_PRIVATE, 1, NULL^C <unfinished ...>
Process 3810 detached


~$ strace -p 3814
Process 3814 attached - interrupt to quit
waitpid(3820, ^C <unfinished ...>
Process 3814 detached

~$ strace -p 3820
Process 3820 attached - interrupt to quit
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
ioctl(0, SNDCTL_TMR_START or TCSETS, {B38400 opost isig -icanon -echo ...}) = ? ERESTARTSYS (To be restarted)
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
ioctl(0, SNDCTL_TMR_START or TCSETS, {B38400 opost isig -icanon -echo ...}) = ? ERESTARTSYS (To be restarted)
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
ioctl(0, SNDCTL_TMR_START or TCSETS, {B38400 opost isig -icanon -echo ...}) = ? ERESTARTSYS (To be restarted)
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
ioctl(0, SNDCTL_TMR_START or TCSETS, {B38400 opost isig -icanon -echo ...}) = ? ERESTARTSYS (To be restarted)
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---

.......
repeating this endlessly

[Bernard]

Thanks for posting strace output.

When compared to Windows I see that Linux has not a good tools for troubleshooting client software - OS interaction.Strace output is not helpful because it does not contain functions callstack.I suppose that one of the ffmpeg thread is constantly and endlessly spinning while possibly waiting on something.Do you have a deadlock?Maybe one of the functions further in call chain did not return?It is very hard to asses what went probably wrong.

[SergeyKostrov]

>>...But Windows is a totally different platform as Ubuntu (Linux). Of course it is and I warned you before. So, do you have sources of the ffmpeg application? If Yes then you could try to add simple logging to a text file in order to understand where the problem is.

[Bernard]

Can you somehow extract from the dump file threads callstacks?Did hang process dump really consume 400mb of committed memory?

[Mark_D_Intel]

We (the IXE development team) can reproduce the hang, but haven't isolated the root cause yet.

[Bernard]

Hi mark,

is there a possibility to dump the hang thread callstack?

[Mark_D_Intel]

There is a change in IXE that runs the target executable in a background process group, and this causes ffmpeg to stop.  A workaround for this case is to redirect input from a file (contents don't matter): inspxe-cl -collect ti-3 -module-filter-mode=include -appdebug=off -app-working-dir (path of ffmpeg-0.6.3)  -- (ffmpeg binary) -threads 2 -i test_10.AVI -target ntsc-dvd -s 1280x720 out_10.MOV < tmp.in

Details:

Ffmpeg adjusts the terminal settings (in init_term, via tcsetattr), apparently to allow interactive exit with 'q'.   The change in terminal settings sends SIGTTOU to the process, which then causes it to stop.   You can verify this behavior by running ffmpeg in the background (independent of IXE) - it will stop (put it in the foreground to resume).   There is some discussion on terminal setting and signals here: http://www.haskell.org/pipermail/glasgow-haskell-bugs/2001-January/000181.html ; (and the spec for tcsetattr here: http://pubs.opengroup.org/onlinepubs/009696899/functions/tcsetattr.html )

[ywsong]

Thanks. It works with the way you suggested.

But it would be nice if the problem is fixed in the next release since users would not know that and running the program that way changes the behavior of the application (in the case of ffmpeg, pressing 'q' does not work if we use the way)

Thanks again for your help.

[Victor_B_2]

Thanks Mark, I've been having the same issue here, but in my case there is no way to make the app I'm profiling non-interactive. I had to download Inspector XE 2011 to make it work.

I second ywsong's words: this looks like a bug or a serious limitation.. it would be nice if you can manage to fix it in later versions.

What is the latest version without this change?