Few issues with mic and mpssd

[Sait_U_]

As I described in another post, mpss-3.2.1 is running on kernel 3.13.10 perfectly (Fedora 20).
 I can run programs on the processor and have no problems except:

1. mpssd daemon  is taking 100% of one host cpu all the time.

2. If mpssd is not started the coprocessor fan is always on. When mpssd starts it goes off when there is no activity. Since people may not want to use the card all the time the default fan speed should be off or low it seems like.

I would like to hear from people running under other confgurations whether mpssd daemon is also taking 100% or not, or is it due to my ocnfiguration.

Thanks

[TaylorIoTKidd]

Hi Sait,

One of our experts suggests you do a "strace -p PID" for mpssd and post.

Regards
--
Taylor
 

[Sait_U_]

OK....all I am seeing is this:

# strace   -p 16931
Process 16931 attached
pause(

with the cursor sitting in front of the bracket. I logged into mic and out but it did not change anything. I waited for about ten minutes. When I ctrl-c I just get the "Process 16931 detached   <detached ...>" message.

[Sait_U_]

I then tried the following

# gdb -p 16931

and issue the "bt" command:(this is with glibc-2.18).

=============================

# gdb -p 16931
GNU gdb (GDB) Fedora 7.6.50.20130731-19.fc20
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".

Attaching to process 16931
Reading symbols from /usr/sbin/mpssd...(no debugging symbols found)...done.
Reading symbols from /lib64/libmpssconfig.so.0.0.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libmpssconfig.so.0.0.1
Reading symbols from /lib64/libscif.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib64/libscif.so.0
Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done.
[New LWP 16934]
[New LWP 16933]
[New LWP 16932]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libgcc_s.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libgcc_s.so.1
0x0000003217c0f07d in pause () from /lib64/libpthread.so.0
Missing separate debuginfos, use: debuginfo-install mpss-daemon-3.2.1-1.glibc2.12.2.x86_64
(gdb) bt
#0  0x0000003217c0f07d in pause () from /lib64/libpthread.so.0
#1  0x000000000040582d in main ()

[BelindaLiviero]

Other than providing a micdebug.sh output to us ( which might have some useful information in the dmesg or /var/log/messages type files) ...if you can actually reliably re-create this problem what I'd suggest you do is restart the mpss, strace -p -f (captured in a typescript session..), cause the problem to happen again (where it becomes runaway)....and send us the typescript capture ...

[Evan_P_Intel]

Sait U. wrote:

Attaching to process 16931

Reading symbols from /usr/sbin/mpssd...(no debugging symbols found)...done.

Reading symbols from /lib64/libmpssconfig.so.0.0.1...(no debugging symbols found)...done.

If you haven't installed the debug-information RPMs for mpssd or the libraries on which it depends, that may be why GDB isn't producing a useful backtrace. If you look in the install tarball that contained the mpss-daemon RPM, you'll see a subdirectory 'dbg' which contains the corresponding mpss-daemon-dbg RPM (and similarly for other x86_64 RPMs). Once these are installed, GDB should be able to find the debugging symbols it needs; hopefully, that will give it (and you) more insight into what mpssd is doing.

[Sait_U_]

This is for Belinda's recommendation:

I am attaching the micdebug file, strace file obtained from "strace -f -o strace.out ./mpss start". I deleted the large part of the strace.out file since it just repeated the same thing that is at the end of the file for 300MB and going. I also noticed that the debug script had a syntax error in running miccheck so I attached that output by running independently. Thansk.

[Sait_U_]

This is regarding Evan's recommendation. I actually did the following: I downloaded the source for mpss-daemon-3.2.1, I then have a spec file that builds the rpm package and the debuginfo package (I am on Fedora 20). I added -g to compile flags and turned off striping of binaries. So, this time when I run gdb -p PID and then "bt" I get:

(gdb) bt
#0  0x0000003217c0f07d in pause () from /lib64/libpthread.so.0
#1  0x0000000000403c22 in start_daemon () at mpssd.c:557
#2  0x0000000000402cc4 in main (argc=1, argv=0x7fffad20e228) at mpssd.c:156

Is this instructive? I don't hae debug symbols from other system libraries but can install them if needed.

[Sait_U_]

OK.....Here is the dbg output with ALL debug packages present:

=======================================================

Attaching to process 9832
Reading symbols from /usr/sbin/mpssd...Reading symbols from /usr/lib/debug/usr/sbin/mpssd.debug...done.
done.
Reading symbols from /lib64/libmpssconfig.so.0.0.1...Reading symbols from /usr/lib/debug/usr/lib64/libmpssconfig.so.0.0.1.debug...done.
done.
Loaded symbols for /lib64/libmpssconfig.so.0.0.1
Reading symbols from /lib64/libscif.so.0...Reading symbols from /usr/lib/debug/usr/lib64/libscif.so.0.0.1.debug...done.
done.
Loaded symbols for /lib64/libscif.so.0
Reading symbols from /lib64/libpthread.so.0...Reading symbols from /usr/lib/debug/usr/lib64/libpthread-2.18.so.debug...done.
done.
[New LWP 9842]
[New LWP 9835]
[New LWP 9834]
[New LWP 9833]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /lib64/libc.so.6...Reading symbols from /usr/lib/debug/usr/lib64/libc-2.18.so.debug...done.
done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from /usr/lib/debug/usr/lib64/ld-2.18.so.debug...done.
done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libgcc_s.so.1...Reading symbols from /usr/lib/debug/usr/lib64/libgcc_s-4.8.2-20131212.so.1.debug...done.
done.
Loaded symbols for /lib64/libgcc_s.so.1
0x0000003217c0f07d in pause () at ../sysdeps/unix/syscall-template.S:81
81      T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS)
(gdb) bt
#0  0x0000003217c0f07d in pause () at ../sysdeps/unix/syscall-template.S:81
#1  0x0000000000403c22 in start_daemon () at mpssd.c:557
#2  0x0000000000402cc4 in main (argc=1, argv=0x7fff51a79938) at mpssd.c:156

===============================================================

[Evan_P_Intel]

Looking at the source code and your "strace -f" log, I have a theory about what the problem is—the code in mic_state() appears to be poll()ing on a sysfs file in a questionable way.

If you're feeling sufficiently adventurous and are willing to verify my supposition, you can apply the attached patch to mpssd.c, recompile mpssd, restart, and see if the CPU usage drops to 0%. The patch is intentionally trivial, but be aware that I've barely tested it.

[Sait_U_]

The patch fixed the problem. After applying mpssd takes 0% and is working just fine. Thanks a lot.