Unable to load action recognition decoder to NCS

[Pan__Alan]

I'm unable to load the Intel model action-recognition-0001-decoder to NCS, either v1 or v2. I'm using the latest release R2 (2019.2.242). However, it can be successfully loaded to CPU (intel core i5) as well as GPU (Gefore GTX 950M).

Here is my minimum (not) working example:

# load_action_decoder.py
import argparse
import os
import sys

from openvino.inference_engine import IENetwork, IEPlugin

if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument('-d', '--device', type=str, default='MYRIAD')
    parser.add_argument('-m', '--model-xml', type=str)
    ARGS = parser.parse_args()

    model_xml = ARGS.model_xml
    model_bin = os.path.splitext(model_xml)[0] + '.bin'
    net = IENetwork(model=model_xml, weights=model_bin)
    net.batch_size = 1

    plugin = IEPlugin(device=ARGS.device)
    exec_net = plugin.load(network=net)

and the error message:

$ python3 load_action_decoder.py --device=MYRIAD --model-xml=/tmp/intel_models/242/Transportation/action_recognition/resnet34_vtn/kinetics/decoder/dldt/FP16/action-recognition-0001-decoder.xml
Traceback (most recent call last):
  File "load_action_decoder.py", line 19, in <module>
    exec_net = plugin.load(network=net)
  File "ie_api.pyx", line 551, in openvino.inference_engine.ie_api.IEPlugin.load
  File "ie_api.pyx", line 561, in openvino.inference_engine.ie_api.IEPlugin.load
RuntimeError: AssertionFailed: inputs.size() == 3

I also tried the previous release R1 (2019.1.144), however, for all device types MYRIAD, CPU, and GPU, it gives me segmentation fault:

python3 load_action_decoder.py --device=MYRIAD --model-xml=/tmp/intel_models/144/Transportation/action_recognition/resnet34_vtn/kinetics/decoder/dldt/action-recognition-0001-decoder-fp16.xml
Segmentation fault (core dumped)

Note that this time I used the model downloaded with R1 downloader.py.

Please help!

[Pan__Alan]

P.S: the same error for model driver-action-recognition-adas-0002-decoder.

[Shubha_R_Intel]

Dear Pan, Alan,

OpenVino has the Action Recognition Demo . Have you tried it on MYRIAD ? Please read the online document and study the example command-line. Let me know if it works for you.

Thanks,

Shubha

[Pan__Alan]

Hi Shubha,

Thanks for the pointer. Yes, the demo itself works. However, by taking a closer look I realized that the demo is actually "cheating":

# /opt/intel/openvino_2019.2.242/deployment_tools/open_model_zoo/demos/python_demos/action_recognition/action_recognition.py#L89
    decoder_target_device = "CPU"
    if args.device != 'CPU':
        encoder_target_device = args.device
    else:
        encoder_target_device = decoder_target_device

It forces the decoder to use CPU. The same error occurs if I use "MYRIAD"  for decoder_target_device as well.

There are some other differences in the demo, such as using IECore instead of IEPlugin, and calling ie.set_config, but they don't seem to help as long as the decoder actually uses MYRIAD.

Please advise.

Thanks!

Alan

[Shubha_R_Intel]

Dear Pan, Alan,

Ha ha. OK I see what you mean. I wonder if the decoder uses some layers unsupported by MYRIAD ? Please look at the Supported Devices Doc . Can you kindly look at the generated IR for action-recognition-0001-decoder  and see if there is an unsupported layer ? If you can't find one, then this is a bug. Whomever wrote the Action Recognition Demo must have "cheated" because he knew that there was a limitation with using the MYRIAD as a detector.

Let me know,

Thanks,

Shubha

[Pan__Alan]

Hi Shubha,

I extracted the layer types from the decoder's xml file, and they all seem to be supported by VPU (except for the Input layer):

$ cat /tmp/intel_models/242/Transportation/action_recognition/resnet34_vtn/kinetics/decoder/dldt/FP16/action-recognition-0001-decoder.xml | grep -oP '(?<=type=").*(?=")' | sort | uniq
Const
Convolution
Eltwise
FullyConnected
GEMM
Input
Permute
Pooling
Power
ReLU
Reshape
SoftMax
Tile

I also tried to programmatically find unsupported layers:

    plugin = IEPlugin(device=ARGS.device)  # MYRIAD
    supported_layers = plugin.get_supported_layers(net)
    not_supported_layers = [l for l in net.layers if l not in supported_layers]
    for l in sorted(not_supported_layers):
        print('{:<30}\t{}'.format(l, net.layers.type))

Some of them did show up, including Const and GEMM types (see below). However, both Const (?!) and GEMM are listed as "supported" by VPU from the documentation. Either the documentation is wrong, or the Python API is wrong, or something else.

0                             	Input
218/Output_0/Data__const      	Const
219/Output_0/Data__const      	Const
220/Output_0/Data__const      	Const
225/Output_0/Data__const      	Const
226/Output_0/Data__const      	Const
227/Output_0/Data__const      	Const
232/Output_0/Data__const      	Const
233/Output_0/Data__const      	Const
234/Output_0/Data__const      	Const
239/Output_0/Data__const      	Const
240/Output_0/Data__const      	Const
241/Output_0/Data__const      	Const
247/Output_0/Data_/copy_const 	Const
249/Output_0/Data_/copy_const 	Const
290                           	GEMM
298                           	GEMM
306                           	GEMM
315                           	GEMM
339                           	GEMM
395                           	GEMM
403                           	GEMM
411                           	GEMM
420                           	GEMM
444                           	GEMM
500                           	GEMM
508                           	GEMM
516                           	GEMM
525                           	GEMM
549                           	GEMM
605                           	GEMM
613                           	GEMM
621                           	GEMM
630                           	GEMM
654                           	GEMM

Is there a workaround or configuration that I can do? If not, when do we expect it to be fixed?

Thanks,
Alan

[Shubha_R_Intel]

Dear Pan, Alan,

I think you may have found a bug.  I have PM'd you to get your version of the action recognition decoder script  where the target decoder device is NCS.

Please send it to me and I  will file a bug on your behalf. 

You can also attach your script as a *.zip to this forum ticket. 

Sorry for the trouble !

Shubha

[Shubha_R_Intel]

Dear Pan, Alan,

Thank you for your cooperation (by sending me your Python script which demos the issue) ! I have filed a bug on your behalf.

Sincerely,

Shubha

[Shubha_R_Intel]

Dear Pan, Alan

You'll be happy to know that this issue has been fixed in OpenVino R3.

Thanks for your patience !

Shubha