topic Thanks Mark, I've been having in Analyzers

Inspector XE hangs on ffmpeg program

ywsong — Sat, 23 Mar 2013 21:10:17 GMT

I just updated Intel Inspector XE to update 5 version.

One of program I was using hangs. It worked fine on previous version.

CPU: Intel(R) Core(TM)2 Duo CPU

OS: Ubuntu 12.04 with kernel 3.2.0-39.

Used ffmpeg-0.6.3 with input file atttached but it does not stop for more than 24 hours.

The exact command line was: (it also hangs on GUI version)

inspxe-cl -collect ti-3 -module-filter-mode=include -appdebug=off -app-working-dir (path of ffmpeg-0.6.3) -- (ffmpeg binary) -threads 2 -i test_10.AVI -target ntsc-dvd -s 1280x720 out_10.MOV

>>...Used ffmpeg-0.6.3 with

SergeyKostrov — Sun, 24 Mar 2013 05:33:03 GMT

>>...Used ffmpeg-0.6.3 with input file atttached but it does not stop for more than 24 hours. Please provide more details for the ffmpeg-0.6.3 program. Is there a link to a download?

ffmpeg is in:

ywsong — Sun, 24 Mar 2013 22:03:52 GMT

ffmpeg is in:

http://www.ffmpeg.org/download.html

They don't have ffmpeg-0.6.3 anymore but you can try it using version 0.6.6.

I think version 0.6.6 has the same problem.

You can see it on the second entry from the bottom of the page.

I see that you're using older

SergeyKostrov — Sat, 27 Apr 2013 05:25:08 GMT

I see that you're using older version of ffmpeg.exe. Next week I'll be able to look at a newer version ( Release configuration ) of ffmpeg.exe: ... ffmpeg version N-50911-g9efcfbe Copyright (c) 2000-2013 the FFmpeg developers built on Mar 13 2013 21:26:48 with gcc 4.7.2 (GCC) ...

@ywsong

Bernard — Tue, 30 Apr 2013 04:54:09 GMT

@ywsong

Is there is a possibility to collect a process dump on Ubuntu?

I dumped the core but the

ywsong — Tue, 30 Apr 2013 20:55:16 GMT

I dumped the core but the size is too big (over 400MB) and it does not allow to attach it.

>>...

SergeyKostrov — Wed, 01 May 2013 01:17:46 GMT

>>... >>OS: Ubuntu 12.04 with kernel 3.2.0-39. >> >>Used ffmpeg-0.6.3 with input file atttached but it does not stop for more than 24 hours. >>... Even if my test is not done yet I will do it on a 64-bit Windows 7 Professional and I hope it doesn't look like useless.

Quote:ywsong wrote:

Bernard — Wed, 01 May 2013 12:52:13 GMT

ywsong wrote:

I dumped the core but the size is too big (over 400MB) and it does not allow to attach it.

Is there an option in GDB to perform an automatic analysis of crashed application?

Quote:iliyapolak wrote:

ywsong — Wed, 01 May 2013 18:25:16 GMT

iliyapolak wrote:

Quote:

ywsongwrote:
I dumped the core but the size is too big (over 400MB) and it does not allow to attach it.

Is there an option in GDB to perform an automatic analysis of crashed application?

It does not crash but hangs.

I looked at it using linux top command and the process was not active.

Sorry of course it hangs.Is

Bernard — Wed, 01 May 2013 19:56:36 GMT

Sorry of course it hangs.Is there any tool on Linux which can analyze hang process? Windows has AD plus.

In your earlier post you mentioned that dump has 400mb of data.Does it contain also kenel mode part?

I couldn't reproduce the

SergeyKostrov — Wed, 01 May 2013 23:40:00 GMT

I couldn't reproduce the problem on a Windows 7 Professional. One thing I've noticed in your command line for the utility is a -thread option. Try to use -thread 1 ( or don't use it at all ) instead of -thread 2.

Quote:Sergey Kostrov wrote:

ywsong — Thu, 02 May 2013 18:13:03 GMT

Sergey Kostrov wrote:

I couldn't reproduce the problem on a Windows 7 Professional. One thing I've noticed in your command line for the utility is a -thread option. Try to use -thread 1 ( or don't use it at all ) instead of -thread 2.

Thanks.

But Windows is a totally different platform as Ubuntu (Linux).

Quote:iliyapolak wrote:

ywsong — Thu, 02 May 2013 18:15:16 GMT

iliyapolak wrote:

Sorry of course it hangs.Is there any tool on Linux which can analyze hang process? Windows has AD plus.

In your earlier post you mentioned that dump has 400mb of data.Does it contain also kenel mode part?

The following is from using strace tool in Linux,

~$ ps -a
PID TTY          TIME CMD
3810 pts/2    00:00:00 inspxe-cl
3814 pts/2    00:00:00 inspxe-runtc
3820 pts/2    00:00:07 ffmpeg_g
3858 pts/1    00:00:00 ps

~$ strace -p 3810
Process 3810 attached - interrupt to quit
futex(0xbfda5e4c, FUTEX_WAIT_PRIVATE, 1, NULL^C <unfinished ...>
Process 3810 detached

~$ strace -p 3814
Process 3814 attached - interrupt to quit
waitpid(3820, ^C <unfinished ...>
Process 3814 detached

~$ strace -p 3820
Process 3820 attached - interrupt to quit
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
ioctl(0, SNDCTL_TMR_START or TCSETS, {B38400 opost isig -icanon -echo ...}) = ? ERESTARTSYS (To be restarted)
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
ioctl(0, SNDCTL_TMR_START or TCSETS, {B38400 opost isig -icanon -echo ...}) = ? ERESTARTSYS (To be restarted)
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
ioctl(0, SNDCTL_TMR_START or TCSETS, {B38400 opost isig -icanon -echo ...}) = ? ERESTARTSYS (To be restarted)
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
ioctl(0, SNDCTL_TMR_START or TCSETS, {B38400 opost isig -icanon -echo ...}) = ? ERESTARTSYS (To be restarted)
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---

.......
repeating this endlessly

Thanks for posting strace

Bernard — Thu, 02 May 2013 19:55:08 GMT

Thanks for posting strace output.

When compared to Windows I see that Linux has not a good tools for troubleshooting client software - OS interaction.Strace output is not helpful because it does not contain functions callstack.I suppose that one of the ffmpeg thread is constantly and endlessly spinning while possibly waiting on something.Do you have a deadlock?Maybe one of the functions further in call chain did not return?It is very hard to asses what went probably wrong.

>>...But Windows is a totally

SergeyKostrov — Fri, 03 May 2013 04:50:35 GMT

>>...But Windows is a totally different platform as Ubuntu (Linux). Of course it is and I warned you before. So, do you have sources of the ffmpeg application? If Yes then you could try to add simple logging to a text file in order to understand where the problem is.

Can you somehow extract from

Bernard — Mon, 06 May 2013 07:12:29 GMT

Can you somehow extract from the dump file threads callstacks?Did hang process dump really consume 400mb of committed memory?

We (the IXE development team)

Mark_D_Intel — Mon, 06 May 2013 13:25:13 GMT

We (the IXE development team) can reproduce the hang, but haven't isolated the root cause yet.

Hi mark,

Bernard — Mon, 06 May 2013 15:56:35 GMT

Hi mark,

is there a possibility to dump the hang thread callstack?

There is a change in IXE that

Mark_D_Intel — Mon, 06 May 2013 22:29:41 GMT

There is a change in IXE that runs the target executable in a background process group, and this causes ffmpeg to stop. A workaround for this case is to redirect input from a file (contents don't matter): inspxe-cl -collect ti-3 -module-filter-mode=include -appdebug=off -app-working-dir (path of ffmpeg-0.6.3) -- (ffmpeg binary) -threads 2 -i test_10.AVI -target ntsc-dvd -s 1280x720 out_10.MOV < tmp.in

Details:

Ffmpeg adjusts the terminal settings (in init_term, via tcsetattr), apparently to allow interactive exit with 'q'. The change in terminal settings sends SIGTTOU to the process, which then causes it to stop. You can verify this behavior by running ffmpeg in the background (independent of IXE) - it will stop (put it in the foreground to resume). There is some discussion on terminal setting and signals here: http://www.haskell.org/pipermail/glasgow-haskell-bugs/2001-January/000181.html ; (and the spec for tcsetattr here: http://pubs.opengroup.org/onlinepubs/009696899/functions/tcsetattr.html )

Thanks. It works with the way

ywsong — Tue, 07 May 2013 18:10:52 GMT

Thanks. It works with the way you suggested.

But it would be nice if the problem is fixed in the next release since users would not know that and running the program that way changes the behavior of the application (in the case of ffmpeg, pressing 'q' does not work if we use the way)

Thanks again for your help.