[Error] mic0: State failed - non existent MIC device

[Munasinghe__Indula]

Hi, 

I have a system with Intel Xeon Phi 3120A coprocessor running with Intel Pentium Gold G5400 processor on a Asus Z370P motherboard with Above 4G decoding enabled. The system was working fine. After an unexpected power cut the coprocessor is not identified by the system. I do not know if the coprocessor is damaged. The coprocessor, blue LED is working as usual and so does the cooling fan. 

When I entered, micctrl -s following result is shown, 

[root@localhost ~]# micctrl -s
  [Error] mic0: State failed - non existent MIC device

When I do miccheck, 

[root@localhost ~]# miccheck
MicCheck 3.8.3-1
Copyright (c) 2016, Intel Corporation.

Executing default tests for host
  Test 0: Check number of devices the OS sees in the system ... pass
  Test 1: Check mic driver is loaded ... fail
    mic driver not loaded

Status: FAIL
Failure: mic driver not loaded

When I do lspci, 

[root@localhost ~]# lspci
01:00.0 Co-processor: Intel Corporation Xeon Phi coprocessor 3120 series (rev 20)

I already tried re-installing the MPSS 3.8.3 and it didn't work. The system was working fine with MPSS 3.8.3. It seems the MIC driver is not loaded. But I do not know how to re-install it on the system. 

I'm very new to these type of systems though I got the system to working state with the help of this forum and the instruction manuals.

Could you please explain how to resolve this problem? 

[JJK]

Which OS are you using? And more specifically, which kernel version (post the output of 'uname -a')?

It could very well be that you are running RHEL/CentOS 7 and have received a kernel update for which Intel has not published a mic driver version.

HTH,

JJK

[Munasinghe__Indula]

Hi, 

Thank you for the reply. 

After searching many forums until your reply, I also beginning to worry that this might be the issue. My kernel is 3.10.0-1062 which is CentOS 7.7. What is the latest CentOS version that MPSS supports ? 

[Loc_N_Intel]

Hi Indula,

MPSS has many release versions. If you use MPSS 3.8 (the latest release version), it supports RedHat 7.3.

[Munasinghe__Indula]

Hi Nguyen, 

Thank you for your Answer. 

[JJK]

You should upgrade to MPSS 3.8.6 to address CVE-2020-0563

As for the RHEL/CentOS kernel version:  I use CentOS 7.7 + mpss 3.8.6 + a patched mpss-modules package and it runs fine. You can find my patches to the mpss-modules package at

  https://www.nikhef.nl/~janjust/mpss/

including a pre-built package for the 3.10.0-1062 kernel.

HTH,

JJK

[Munasinghe__Indula]

Hi 

My problem is not solved. I reinstalled CentOS 7.3. But I can't get to rebuild the mpss rpm files. 

The bash is as follows. 

[root@localhost src]# ls
dapl-2.1.9-2.glibc2.12.src.rpm  mpss-modules-3.8.6-1.src.rpm
libibscif-1.0.0-1.el6.src.rpm   ofed-driver-1.0.0-1.el6.src.rpm

[root@localhost src]# rpmbuild --rebuild mpss-modules*.src.rpm
Installing mpss-modules-3.8.6-1.src.rpm
warning: user qb_user does not exist - using root
warning: group qb_user does not exist - using root
warning: user qb_user does not exist - using root
warning: group qb_user does not exist - using root
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.ipFa6b
+ umask 022
+ cd /root/rpmbuild/BUILD
+ cd /root/rpmbuild/BUILD
+ rm -rf mpss-modules-3.8.6
+ /usr/bin/mkdir -p mpss-modules-3.8.6
+ cd mpss-modules-3.8.6
+ /usr/bin/bzip2 -dc /root/rpmbuild/SOURCES/mpss-modules-3.8.6.tar.bz2
+ /usr/bin/tar -xvvf -
drwxrwxr-x qb_user/qb_user   0 2019-07-25 13:09 ./
-rw-rw-r-- qb_user/qb_user 411 2019-07-01 16:45 ./udev-mic.rules
-rw-rw-r-- qb_user/qb_user  49 2019-07-01 16:45 ./.mpss-metadata
drwxrwxr-x qb_user/qb_user   0 2019-07-01 16:45 ./dma/
-rw-rw-r-- qb_user/qb_user 120 2019-07-01 16:45 ./dma/Kbuild
-rw-rw-r-- qb_user/qb_user 16015 2019-07-01 16:45 ./dma/mic_dma_md.c
-rw-rw-r-- qb_user/qb_user  2151 2019-07-01 16:45 ./dma/mic_sbox_md.c
-rw-rw-r-- qb_user/qb_user 54732 2019-07-01 16:45 ./dma/mic_dma_lib.c
-rw-rw-r-- qb_user/qb_user  3589 2019-07-01 16:45 ./Kbuild
drwxrwxr-x qb_user/qb_user     0 2019-07-01 16:45 ./mpssboot/
-rw-rw-r-- qb_user/qb_user  6910 2019-07-01 16:45 ./mpssboot/mpssboot.c
-rw-rw-r-- qb_user/qb_user    20 2019-07-01 16:45 ./mpssboot/Kbuild
drwxrwxr-x qb_user/qb_user     0 2019-07-01 16:45 ./pm_scif/
-rw-rw-r-- qb_user/qb_user    19 2019-07-01 16:45 ./pm_scif/Kbuild
-rw-rw-r-- qb_user/qb_user 10092 2019-07-01 16:45 ./pm_scif/pm_scif.c
-rw-rw-r-- qb_user/qb_user  2152 2019-07-01 16:45 ./pm_scif/pm_scif.h
drwxrwxr-x qb_user/qb_user     0 2019-07-01 16:45 ./ras/
-rw-rw-r-- qb_user/qb_user 32564 2019-07-01 16:45 ./ras/micras_uncore.c
-rw-rw-r-- qb_user/qb_user   207 2019-07-01 16:45 ./ras/Kbuild
-rw-rw-r-- qb_user/qb_user 68817 2019-07-01 16:45 ./ras/micras_main.c
-rw-rw-r-- qb_user/qb_user 32895 2019-07-01 16:45 ./ras/micras_api.h
-rw-rw-r-- qb_user/qb_user  6557 2019-07-01 16:45 ./ras/monahan.h
-rw-rw-r-- qb_user/qb_user 37460 2019-07-01 16:45 ./ras/micras_knf.c
-rw-rw-r-- qb_user/qb_user 77928 2019-07-01 16:45 ./ras/micras_elog.c
-rw-rw-r-- qb_user/qb_user 69571 2019-07-01 16:45 ./ras/micras_knc.c
-rw-rw-r-- qb_user/qb_user  6491 2019-07-01 16:45 ./ras/Makefile
-rw-rw-r-- qb_user/qb_user 30533 2019-07-01 16:45 ./ras/micras_core.c
-rw-rw-r-- qb_user/qb_user 24090 2019-07-01 16:45 ./ras/micras_common.c
-rw-rw-r-- qb_user/qb_user 28421 2019-07-01 16:45 ./ras/micras_pm.c
-rw-rw-r-- qb_user/qb_user 10224 2019-07-01 16:45 ./ras/micpm_api.h
-rw-rw-r-- qb_user/qb_user 14455 2019-07-01 16:45 ./ras/micras.h
-rw-rw-r-- qb_user/qb_user  4687 2019-07-01 16:45 ./ras/micmca_api.h
drwxrwxr-x qb_user/qb_user     0 2019-07-01 16:45 ./trace_capture/
-rw-rw-r-- qb_user/qb_user    25 2019-07-01 16:45 ./trace_capture/Kbuild
-rw-rw-r-- qb_user/qb_user 10647 2019-07-01 16:45 ./trace_capture/tc_host.c
-rw-rw-r-- qb_user/qb_user  2415 2019-07-01 16:45 ./trace_capture/docapture.c
-rw-rw-r-- qb_user/qb_user  8602 2019-07-01 16:45 ./trace_capture/trace_capture.h
-rw-rw-r-- qb_user/qb_user   569 2019-07-01 16:45 ./trace_capture/Makefile
-rw-rw-r-- qb_user/qb_user  2816 2019-07-01 16:45 ./trace_capture/tc_memcvt.c
-rw-rw-r-- qb_user/qb_user 68560 2019-07-01 16:45 ./trace_capture/trace_capture.c
drwxrwxr-x qb_user/qb_user     0 2019-07-01 16:45 ./micscif/
-rw-rw-r-- qb_user/qb_user 11996 2019-07-01 16:45 ./micscif/micscif_rb.c
-rw-rw-r-- qb_user/qb_user 74401 2019-07-01 16:45 ./micscif/micscif_rma.c
-rw-rw-r-- qb_user/qb_user 16182 2019-07-01 16:45 ./micscif/micscif_va_gen.c
-rw-rw-r-- qb_user/qb_user 16374 2019-07-01 16:45 ./micscif/micscif_rma_list.c
-rw-rw-r-- qb_user/qb_user 31378 2019-07-01 16:45 ./micscif/micscif_debug.c
-rw-rw-r-- qb_user/qb_user 12177 2019-07-01 16:45 ./micscif/micscif_select.c
-rw-rw-r-- qb_user/qb_user 44878 2019-07-01 16:45 ./micscif/micscif_nm.c
-rw-rw-r-- qb_user/qb_user   584 2019-07-01 16:45 ./micscif/Kbuild
-rw-rw-r-- qb_user/qb_user  6661 2019-07-01 16:45 ./micscif/micscif_sysfs.c
-rw-rw-r-- qb_user/qb_user 13498 2019-07-01 16:45 ./micscif/micscif_smpt.c
-rw-rw-r-- qb_user/qb_user 96705 2019-07-01 16:45 ./micscif/micscif_api.c
-rw-rw-r-- qb_user/qb_user 18490 2019-07-01 16:45 ./micscif/micscif_main.c
-rw-rw-r-- qb_user/qb_user  5143 2019-07-01 16:45 ./micscif/micscif_intr.c
-rw-rw-r-- qb_user/qb_user 29259 2019-07-01 16:45 ./micscif/micscif_rma_dma.c
-rw-rw-r-- qb_user/qb_user 12840 2019-07-01 16:45 ./micscif/micscif_fd.c
-rw-rw-r-- qb_user/qb_user 85521 2019-07-01 16:45 ./micscif/micscif_nodeqp.c
-rw-rw-r-- qb_user/qb_user  6601 2019-07-01 16:45 ./micscif/micscif_va_node.c
-rw-rw-r-- qb_user/qb_user  8999 2019-07-01 16:45 ./micscif/micscif_ports.c
-rw-rw-r-- qb_user/qb_user 18092 2019-07-01 16:45 ./COPYING
-rw-rw-r-- qb_user/qb_user  4436 2019-07-01 16:45 ./Makefile
-rwxrwxr-x qb_user/qb_user    88 2019-07-01 16:45 ./mic.modules
drwxrwxr-x qb_user/qb_user     0 2019-07-01 16:45 ./include/
-rw-rw-r-- qb_user/qb_user  6481 2019-07-01 16:45 ./include/scif_ioctl.h
-rw-rw-r-- qb_user/qb_user  3744 2019-07-01 16:45 ./include/micint.h
drwxrwxr-x qb_user/qb_user     0 2019-07-01 16:45 ./include/mic/
-rw-rw-r-- qb_user/qb_user  2515 2019-07-01 16:45 ./include/mic/micscif_kmem_cache.h
-rw-rw-r-- qb_user/qb_user 12511 2019-07-01 16:45 ./include/mic/mic_pm.h
-rw-rw-r-- qb_user/qb_user  4050 2019-07-01 16:45 ./include/mic/micveth.h
-rw-rw-r-- qb_user/qb_user  4823 2019-07-01 16:45 ./include/mic/micvcons.h
-rw-rw-r-- qb_user/qb_user  7461 2019-07-01 16:45 ./include/mic/micveth_dma.h
-rw-rw-r-- qb_user/qb_user  3993 2019-07-01 16:45 ./include/mic/micscif_smpt.h
-rw-rw-r-- qb_user/qb_user  8075 2019-07-01 16:45 ./include/mic/micscif_nm.h
-rw-rw-r-- qb_user/qb_user 27176 2019-07-01 16:45 ./include/mic/micscif_rma.h
-rw-rw-r-- qb_user/qb_user  6156 2019-07-01 16:45 ./include/mic/compl_buf_ring.h
-rw-rw-r-- qb_user/qb_user  6023 2019-07-01 16:45 ./include/mic/io_interface.h
-rw-rw-r-- qb_user/qb_user  4631 2019-07-01 16:45 ./include/mic/bootparams.h
-rw-rw-r-- qb_user/qb_user  9223 2019-07-01 16:45 ./include/mic/micscif_nodeqp.h
-rw-rw-r-- qb_user/qb_user 27016 2019-07-01 16:45 ./include/mic/micscif.h
-rw-rw-r-- qb_user/qb_user  2272 2019-07-01 16:45 ./include/mic/micpsmi.h
-rw-rw-r-- qb_user/qb_user  4292 2019-07-01 16:45 ./include/mic/micbaseaddressdefine.h
-rw-rw-r-- qb_user/qb_user  2517 2019-07-01 16:45 ./include/mic/micveth_common.h
-rw-rw-r-- qb_user/qb_user  3220 2019-07-01 16:45 ./include/mic/micscif_va_gen.h
-rw-rw-r-- qb_user/qb_user  4929 2019-07-01 16:45 ./include/mic/micscif_rma_list.h
-rw-rw-r-- qb_user/qb_user  3120 2019-07-01 16:45 ./include/mic/mic_sbox_md.h
-rw-rw-r-- qb_user/qb_user  2635 2019-07-01 16:45 ./include/mic/mic_virtio.h
-rw-rw-r-- qb_user/qb_user  3835 2019-07-01 16:45 ./include/mic/micscif_va_node.h
-rw-rw-r-- qb_user/qb_user  7775 2019-07-01 16:45 ./include/mic/ringbuffer.h
-rw-rw-r-- qb_user/qb_user  2351 2019-07-01 16:45 ./include/mic/micscif_intr.h
-rw-rw-r-- qb_user/qb_user 14550 2019-07-01 16:45 ./include/mic/mic_dma_md.h
-rw-rw-r-- qb_user/qb_user  2108 2019-07-01 16:45 ./include/mic/micdboxdefine.h
-rw-rw-r-- qb_user/qb_user  7436 2019-07-01 16:45 ./include/mic/mic_dma_lib.h
-rw-rw-r-- qb_user/qb_user  5778 2019-07-01 16:45 ./include/mic/micscif_rb.h
-rw-rw-r-- qb_user/qb_user  7402 2019-07-01 16:45 ./include/mic/micscif_map.h
-rw-rw-r-- qb_user/qb_user 10281 2019-07-01 16:45 ./include/mic/micsboxdefine.h
-rw-rw-r-- qb_user/qb_user  4215 2019-07-01 16:45 ./include/mic/mic_macaddr.h
-rw-rw-r-- qb_user/qb_user  6544 2019-07-01 16:45 ./include/mic/mic_dma_api.h
-rw-rw-r-- qb_user/qb_user  3953 2019-07-01 16:45 ./include/mic_interrupts.h
-rw-rw-r-- qb_user/qb_user 22586 2019-07-01 16:45 ./include/mic_common.h
-rw-rw-r-- qb_user/qb_user 68604 2019-07-01 16:45 ./include/scif.h
drwxrwxr-x qb_user/qb_user     0 2019-07-01 16:45 ./virtio/
-rw-rw-r-- qb_user/qb_user    24 2019-07-01 16:45 ./virtio/Kbuild
-rw-rw-r-- qb_user/qb_user 22334 2019-07-01 16:45 ./virtio/mic_virtblk.c
drwxrwxr-x qb_user/qb_user     0 2019-07-01 16:45 ./vcons/
-rw-rw-r-- qb_user/qb_user    44 2019-07-01 16:45 ./vcons/Kbuild
-rw-rw-r-- qb_user/qb_user  3737 2019-07-01 16:45 ./vcons/hvc_console.h
-rw-rw-r-- qb_user/qb_user  9616 2019-07-01 16:45 ./vcons/hvc_mic.c
drwxrwxr-x qb_user/qb_user     0 2019-07-01 16:45 ./ramoops/
-rw-rw-r-- qb_user/qb_user  4216 2019-07-01 16:45 ./ramoops/ramoops.c
-rw-rw-r-- qb_user/qb_user    19 2019-07-01 16:45 ./ramoops/Kbuild
-rw-rw-r-- qb_user/qb_user  1334 2019-07-01 16:45 ./mic.conf
drwxrwxr-x qb_user/qb_user     0 2019-07-01 16:45 ./host/
-rw-rw-r-- qb_user/qb_user 17157 2019-07-01 16:45 ./host/pm_ioctl.c
-rw-rw-r-- qb_user/qb_user 28523 2019-07-01 16:45 ./host/micscif_pm.c
-rw-rw-r-- qb_user/qb_user 21744 2019-07-01 16:45 ./host/linvnet.c
-rw-rw-r-- qb_user/qb_user  4940 2019-07-01 16:45 ./host/linpsmi.c
-rw-rw-r-- qb_user/qb_user 19188 2019-07-01 16:45 ./host/linvcons.c
-rw-rw-r-- qb_user/qb_user 29774 2019-07-01 16:45 ./host/pm_pcstate.c
-rw-rw-r-- qb_user/qb_user 29536 2019-07-01 16:45 ./host/tools_support.c
-rw-rw-r-- qb_user/qb_user  9705 2019-07-01 16:45 ./host/linscif_host.c
-rw-rw-r-- qb_user/qb_user  1444 2019-07-01 16:45 ./host/Makefile
-rw-rw-r-- qb_user/qb_user 22761 2019-07-01 16:45 ./host/linsysfs.c
drwxrwxr-x qb_user/qb_user     0 2019-07-01 16:45 ./host/vhost/
-rw-rw-r-- qb_user/qb_user  7930 2019-07-01 16:45 ./host/vhost/vhost.h
-rw-rw-r-- qb_user/qb_user 18710 2019-07-01 16:45 ./host/vhost/mic_vhost.c
-rw-rw-r-- qb_user/qb_user 16910 2019-07-01 16:45 ./host/vhost/mic_blk.c
-rw-rw-r-- qb_user/qb_user  6527 2019-07-01 16:45 ./host/linpm.c
-rw-rw-r-- qb_user/qb_user  5404 2019-07-01 16:45 ./host/ioctl.c
-rw-rw-r-- qb_user/qb_user 22242 2019-07-01 16:45 ./host/linux.c
-rw-rw-r-- qb_user/qb_user 22798 2019-07-01 16:45 ./host/vmcore.c
-rw-rw-r-- qb_user/qb_user 53803 2019-07-01 16:45 ./host/uos_download.c
-rw-rw-r-- qb_user/qb_user  5386 2019-07-01 16:45 ./host/acptboot.c
-rw-rw-r-- qb_user/qb_user  6154 2019-07-01 16:45 ./host/micpsmi.c
drwxrwxr-x qb_user/qb_user     0 2019-07-01 16:45 ./vnet/
-rw-rw-r-- qb_user/qb_user    88 2019-07-01 16:45 ./vnet/Kbuild
-rw-rw-r-- qb_user/qb_user  3205 2019-07-01 16:45 ./vnet/mic.h
-rw-rw-r-- qb_user/qb_user 43086 2019-07-01 16:45 ./vnet/micveth_dma.c
-rw-rw-r-- qb_user/qb_user  3320 2019-07-01 16:45 ./vnet/micveth_param.c
-rw-rw-r-- qb_user/qb_user 24018 2019-07-01 16:45 ./vnet/micveth.c
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .
+ exit 0
Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.i2AIuc
+ umask 022
+ cd /root/rpmbuild/BUILD
+ cd mpss-modules-3.8.6
+ '[' -d modules ']'
+ /usr/bin/make -j4 KERNEL_VERSION=3.10.0-514.el7.x86_64 KERNEL_SRC=/lib/modules/3.10.0-514.el7.x86_64/build CC=gcc 'LD=ld ' MIC_CARD_ARCH=k1om
/usr/bin/make -C /lib/modules/3.10.0-514.el7.x86_64/build M=/root/rpmbuild/BUILD/mpss-modules-3.8.6 modules \
	INSTALL_MOD_PATH=
make: *** /lib/modules/3.10.0-514.el7.x86_64/build: No such file or directory.  Stop.
make: *** [modules] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.i2AIuc (%build)


RPM build errors:
    user qb_user does not exist - using root
    group qb_user does not exist - using root
    user qb_user does not exist - using root
    group qb_user does not exist - using root
    Bad exit status from /var/tmp/rpm-tmp.i2AIuc (%build)

What's a qb_user? I've extracted the files using root. 

[Munasinghe__Indula]

Above problem was recovered after installing gcc in the system.