Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6392 Discussions

Myriad plugin and multipule Myriad X cards/devices

OKI__HIdetaka
New Contributor I
705 Views

Hi,

I see a problem to use myriad plugin for multiple Myriad X - two M.2 cards and two Myriad X on each card, total four devices.

I can not use HDDL plugin or hddl utilities on the host since it is arm64 box and dldt is built from GitHub souce.  Hddl module is not open sourced.    Dldt version is R1.1 which is latest on GitHub.

I can use two myriad Xs but got errors to invoke third process.  I can invoke eight processes on the xeon server with one PCI card which has eight Myriad X to use myriad plugin. I am wondering there would be some issue to deal with multiple cards.

Here is the error symptom.  I will appreciate any suggestion, comment and so on.

--------

<<<< There were four devices at first. >>>>

nfb@box3:~$ lsusb
Bus 001 Device 002: ID 0424:2514 Standard Microsystems Corp. USB 2.0 Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 007 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 006 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 005 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 004 Device 004: ID 03e7:2485
Bus 004 Device 003: ID 2c42:5114
Bus 004 Device 002: ID 03e7:2485
Bus 004 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 002 Device 004: ID 03e7:2485
Bus 002 Device 003: ID 2c42:5114
Bus 002 Device 002: ID 03e7:2485
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

<<<< Then invoked infrence process for Myriad X. >>>>


[ INFO ] Preparing output blobs
[ INFO ] Loading model to the plugin
[ INFO ] Starting inference (8192 iterations)
[ INFO ] Processing output blobs

<<<< Then invoked send process. >>>>

[ INFO ] Preparing output blobs
[ INFO ] Loading model to the plugin
E: [ncAPI] [    399818] resetAll:334     Failed to connect to stalled device, rc: X_LINK_ERROR
[ INFO ] Starting inference (8192 iterations)
[ INFO ] Processing output blobs

<<<< Then invoked third process. >>>>

[ INFO ] Preparing output blobs
[ INFO ] Loading model to the plugin
E: [ncAPI] [    406668] resetAll:334     Failed to connect to stalled device, rc: X_LINK_ERROR
E: [ncAPI] [    406668] resetAll:334     Failed to connect to stalled device, rc: X_LINK_ERROR
E: [xLink] [    426510] dispatcherWaitEventComplete:694 waiting is timeout, sending reset remote event
E: [xLink] [    429515] dispatcherEventReceive:333      dispatcherEventReceive() Read failed -4 | event 0x7f877fe990 XLINK_WRITE_REQ

E: [xLink] [    429516] eventReader:219 eventReader stopped
E: [xLink] [    429524] dispatcherEventSend:910 Write failed header -7 | event XLINK_WRITE_REQ

E: [xLink] [    429525] eventSchedulerRun:570   Event sending failed
E: [xLink] [    429525] dispatcherEventSend:910 Write failed header -4 | event XLINK_RESET_REQ

E: [xLink] [    429525] eventSchedulerRun:570   Event sending failed
E: [watchdog] [    429526] sendPingMessage:120  Failed send ping message: X_LINK_TIMEOUT
E: [watchdog] [    429527] sendPingMessage:120  Failed send ping message: X_LINK_ERROR
E: [watchdog] [    429527] watchdog_routine:315 [0x55a4423290] device, not respond, removing from watchdog

[ INFO ] Starting inference (8192 iterations)
E: [ncAPI] [    429550] ncFifoDestroy:3162      Failed to write to fifo before deleting it!
E: [ncAPI] [    439551] ncDeviceClose:1603      Device didn't appear after reboot
[ ERROR ] Failed to queue inference: NC_ERROR

<<<< NC_ERROR occured. >>>>

foo@box3:~$ lsusb
Bus 001 Device 002: ID 0424:2514 Standard Microsystems Corp. USB 2.0 Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 007 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 006 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 005 Device 002: ID 03e7:f63b
Bus 005 Device 003: ID 03e7:f63b
Bus 005 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 004 Device 003: ID 2c42:5114
Bus 004 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

<<<< Two devices became USB3.0 ID f63b.  Two devices on 2nd card vanished. >>>>

[ INFO ] Execution successful

[ INFO ] Execution successful

<<<< The first and second process completed. >>>>

foo@box3:~$ lsusb
Bus 001 Device 002: ID 0424:2514 Standard Microsystems Corp. USB 2.0 Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 007 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 006 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 005 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 004 Device 005: ID 03e7:2485
Bus 004 Device 003: ID 2c42:5114
Bus 004 Device 006: ID 03e7:2485
Bus 004 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

<<<< Two devices came back to USB2.0 ID 2485. The second card kept vanishiing. >>>>

After rebooting all deives were found by lsusb.

--------

 

0 Kudos
1 Reply
Shubha_R_Intel
Employee
705 Views

Dear OKI, HIdetaka

Most likely you need to set the KEY_VPU_FORCE_RESET option of Myriad plugin to NO: https://docs.openvinotoolkit.org/2019_R1.1/_docs_IE_DG_supported_plugins_MYRIAD.html#supported_configuration_parameters

It defaults to NO in R2 but you are trying R1.1 where the default is YES. Anyway running multiple processes at the same time requires this option to be NO. So even if this is not the root cause of your issue, you  still need KEY_VPU_FORCE_RESET option of Myriad plugin to be set to NO.

Hope it helps,

Thanks,

Shubha

 

0 Kudos
Reply