OpenVINO, virsh and NCS2

John8 · ‎06-10-2019

Hello,

I am trying to use virsh, OpenVINO and NCS2 together. CentOS7 is installed on the host. On virtual machine I have Ubuntu 16.04.6 LTS and openvino(2019_R1.1).
When I run on virtual machine "./demo_squeezenet_download_convert_run.sh -d MYRIAD" or
"./benchmark_app -i ~/TEST/ -d MYRIAD -m /root/openvino_models/ir/FP16/classification/squeezenet/1.1/caffe/squeezenet1.1.xml", I get the error:

...
[Step 5/8] Loading model to the plugin
I: [ncAPI] [     33089] ncDeviceOpen:738        ncDeviceOpen() XLinkBootRemote is running for 2.1-ma2480...

I: [ncAPI] [     33390] ncDeviceOpen:754        ncDeviceOpen() XLinkBootRemote returned success X_LINK_SUCCESS for 2.1-ma2480
E: [ncAPI] [     48390] ncDeviceOpen:800        Failed connection to device (2.1-ma2480) with error 5
I: [ncAPI] [     48390] destroyDeviceHandle:1439        Destroying device handler
I: [ncAPI] [     48391] ncDeviceClose:1475      Handle already destroyed
[ ERROR ] Can not init USB device: NC_DEVICE_NOT_FOUND

I think the problem exists because the device requires redirection of two usb ports (for 03e7:2485 and for 03e7:f63b), but only one of them can be available. I can do usb pass-through from host to virtual machine for 03e7:2485 but not for 03e7:f63b, because this second device is not visible.

I tried to solve this problem by adding the rules in /etc/udev/rules.d/98-libvirt-usb.rules:

ACTION=="add", \
    SUBSYSTEM=="usb", \
    ENV{ID_VENDOR_ID}=="03e7", \
    ENV{ID_MODEL_ID}=="2485", \
    RUN+="virsh attach-device virtual_machine /etc/usb2-device.xml"
ACTION=="remove", \
    SUBSYSTEM=="usb", \
    ENV{ID_VENDOR_ID}=="03e7", \
    ENV{ID_MODEL_ID}=="2485", \
    RUN+="virsh detach-device virtual_machine /etc/usb2-device.xml"
ACTION=="add", \
    SUBSYSTEM=="usb", \
    ENV{ID_VENDOR_ID}=="03e7", \
    ENV{ID_MODEL_ID}=="f63b", \
    RUN+="virsh attach-device virtual_machine /etc/usb3-device.xml"
ACTION=="remove", \
    SUBSYSTEM=="usb", \
    ENV{ID_VENDOR_ID}=="03e7", \
    ENV{ID_MODEL_ID}=="f63b", \
    RUN+="virsh detach-device virtual_machine /etc/usb3-device.xml"

cat /etc/usb2-device.xml

    <hostdev mode='subsystem' type='usb' managed='yes'>
      <source>
        <vendor id='0x03e7'/>
        <product id='0x2485'/>
      </source>
      <address type='usb' bus='1' port='1'/>
    </hostdev>

cat /etc/usb3-device.xml

    <hostdev mode='subsystem' type='usb' managed='yes'>
      <source>
        <vendor id='0x03e7'/>
        <product id='0xf63b'/>
      </source>
    </hostdev>

But that did not solve the problem.

Below you can find what is returned by the "udevadm monitor" command (on host and on virtual machine) when I run benchmark_app on virtual machine.

"udevadm monitor" output (on host):

KERNEL[294693.827080] remove   /devices/pci0000:00/0000:00:14.0/usb1/1-2/1-2:1.0 (usb)
KERNEL[294693.829032] remove   /devices/pci0000:00/0000:00:14.0/usb1/1-2 (usb)
UDEV  [294693.836708] remove   /devices/pci0000:00/0000:00:14.0/usb1/1-2/1-2:1.0 (usb)
UDEV  [294693.923573] remove   /devices/pci0000:00/0000:00:14.0/usb1/1-2 (usb)
KERNEL[294694.226517] add      /devices/pci0000:00/0000:00:14.0/usb2/2-8 (usb)
KERNEL[294694.228549] add      /devices/pci0000:00/0000:00:14.0/usb2/2-8/2-8:1.0 (usb)
UDEV  [294694.327234] add      /devices/pci0000:00/0000:00:14.0/usb2/2-8 (usb)
UDEV  [294694.335584] add      /devices/pci0000:00/0000:00:14.0/usb2/2-8/2-8:1.0 (usb)
KERNEL[294694.575255] remove   /devices/pci0000:00/0000:00:14.0/usb2/2-8/2-8:1.0 (usb)
KERNEL[294694.575973] remove   /devices/pci0000:00/0000:00:14.0/usb2/2-8 (usb)
UDEV  [294694.576449] remove   /devices/pci0000:00/0000:00:14.0/usb2/2-8/2-8:1.0 (usb)
UDEV  [294694.621523] remove   /devices/pci0000:00/0000:00:14.0/usb2/2-8 (usb)
KERNEL[294694.876700] add      /devices/pci0000:00/0000:00:14.0/usb1/1-2 (usb)
KERNEL[294694.877105] add      /devices/pci0000:00/0000:00:14.0/usb1/1-2/1-2:1.0 (usb)
UDEV  [294694.968786] add      /devices/pci0000:00/0000:00:14.0/usb1/1-2 (usb)
UDEV  [294694.975484] add      /devices/pci0000:00/0000:00:14.0/usb1/1-2/1-2:1.0 (usb)

"udevadm monitor" output (on virtual machine):

KERNEL[5721.180787] remove   /devices/pci0000:00/0000:00:07.0/usb2/2-1/2-1:1.0 (usb)
UDEV  [5721.181086] remove   /devices/pci0000:00/0000:00:07.0/usb2/2-1/2-1:1.0 (usb)
KERNEL[5721.181340] remove   /devices/pci0000:00/0000:00:07.0/usb2/2-1 (usb)
UDEV  [5721.181580] remove   /devices/pci0000:00/0000:00:07.0/usb2/2-1 (usb)
KERNEL[5724.243200] add      /devices/pci0000:00/0000:00:07.0/usb2/2-1 (usb)
UDEV  [5724.247746] add      /devices/pci0000:00/0000:00:07.0/usb2/2-1 (usb)
KERNEL[5724.248909] add      /devices/pci0000:00/0000:00:07.0/usb2/2-1/2-1:1.0 (usb)
UDEV  [5724.251771] add      /devices/pci0000:00/0000:00:07.0/usb2/2-1/2-1:1.0 (usb)

It looks like the usb devices on the host are switching between themselves, but not on the virtual machine. Maybe in the /etc/usb3-device.xml file I should add <address type='usb' bus='' port=''/>? But I do not know what values should be set for bus and port.

In usb2-device.xml file (for 0x03e7:0x2485) I had to add <address type='usb' bus='1' port='1'/>. Without this, the device was connected to the controller usb1 instead of to usb2. But I don't know why for 'bus' I had to set 1. Below you can check that according to 'lsusb -t' the device is connected to bus 2, not to bus 1.What values (bus and port) should be valid for the 0x03e7:0xf63b device? This device should be connected to the usb3 controller.

On virtual machine lsusb -t returns:

/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 480M
    |__ Port 1: Dev 5, If 0, Class=Vendor Specific Class, Driver=, 480M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=uhci_hcd/2p, 12M

How can I fix this problem with virsh and NCS2?