Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6403 Discussions

Unable to load action recognition decoder to NCS

Pan__Alan
Beginner
‎08-19-2019 11:06 AM
800 Views

I'm unable to load the Intel model action-recognition-0001-decoder to NCS, either v1 or v2. I'm using the latest release R2 (2019.2.242). However, it can be successfully loaded to CPU (intel core i5) as well as GPU (Gefore GTX 950M).

Here is my minimum (not) working example:

# load_action_decoder.py
import argparse
import os
import sys

from openvino.inference_engine import IENetwork, IEPlugin

if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument('-d', '--device', type=str, default='MYRIAD')
    parser.add_argument('-m', '--model-xml', type=str)
    ARGS = parser.parse_args()

    model_xml = ARGS.model_xml
    model_bin = os.path.splitext(model_xml)[0] + '.bin'
    net = IENetwork(model=model_xml, weights=model_bin)
    net.batch_size = 1

    plugin = IEPlugin(device=ARGS.device)
    exec_net = plugin.load(network=net)

and the error message:

$ python3 load_action_decoder.py --device=MYRIAD --model-xml=/tmp/intel_models/242/Transportation/action_recognition/resnet34_vtn/kinetics/decoder/dldt/FP16/action-recognition-0001-decoder.xml
Traceback (most recent call last):
  File "load_action_decoder.py", line 19, in <module>
    exec_net = plugin.load(network=net)
  File "ie_api.pyx", line 551, in openvino.inference_engine.ie_api.IEPlugin.load
  File "ie_api.pyx", line 561, in openvino.inference_engine.ie_api.IEPlugin.load
RuntimeError: AssertionFailed: inputs.size() == 3

I also tried the previous release R1 (2019.1.144), however, for all device types MYRIAD, CPU, and GPU, it gives me segmentation fault:

python3 load_action_decoder.py --device=MYRIAD --model-xml=/tmp/intel_models/144/Transportation/action_recognition/resnet34_vtn/kinetics/decoder/dldt/action-recognition-0001-decoder-fp16.xml
Segmentation fault (core dumped)

Note that this time I used the model downloaded with R1 downloader.py.

Please help!

0 Kudos

Link Copied

8 Replies
Pan__Alan
Beginner
‎08-19-2019 11:17 AM
800 Views

P.S: the same error for model driver-action-recognition-adas-0002-decoder.

0 Kudos
Copy link
Shubha_R_Intel
Employee
‎08-19-2019 02:12 PM
800 Views

Dear Pan, Alan,

OpenVino has the Action Recognition Demo . Have you tried it on MYRIAD ? Please read the online document and study the example command-line. Let me know if it works for you.

Thanks,

Shubha

 

0 Kudos
Copy link
Pan__Alan
Beginner
‎08-19-2019 03:01 PM
800 Views

Hi Shubha,

Thanks for the pointer. Yes, the demo itself works. However, by taking a closer look I realized that the demo is actually "cheating":

# /opt/intel/openvino_2019.2.242/deployment_tools/open_model_zoo/demos/python_demos/action_recognition/action_recognition.py#L89
    decoder_target_device = "CPU"
    if args.device != 'CPU':
        encoder_target_device = args.device
    else:
        encoder_target_device = decoder_target_device

It forces the decoder to use CPU. The same error occurs if I use "MYRIAD"  for decoder_target_device as well.

There are some other differences in the demo, such as using IECore instead of IEPlugin, and calling ie.set_config, but they don't seem to help as long as the decoder actually uses MYRIAD.

Please advise.

Thanks!

Alan

 

0 Kudos
Copy link
Shubha_R_Intel
Employee
‎08-20-2019 03:13 PM
800 Views

Dear Pan, Alan,

Ha ha. OK I see what you mean. I wonder if the decoder uses some layers unsupported by MYRIAD ? Please look at the Supported Devices Doc . Can you kindly look at the generated IR for action-recognition-0001-decoder  and see if there is an unsupported layer ? If you can't find one, then this is a bug. Whomever wrote the Action Recognition Demo must have "cheated" because he knew that there was a limitation with using the MYRIAD as a detector.

Let me know,

Thanks,

Shubha

0 Kudos
Copy link
Pan__Alan
Beginner
‎08-20-2019 05:41 PM
800 Views

Hi Shubha,

I extracted the layer types from the decoder's xml file, and they all seem to be supported by VPU (except for the Input layer):

$ cat /tmp/intel_models/242/Transportation/action_recognition/resnet34_vtn/kinetics/decoder/dldt/FP16/action-recognition-0001-decoder.xml | grep -oP '(?<=type=").*(?=")' | sort | uniq
Const
Convolution
Eltwise
FullyConnected
GEMM
Input
Permute
Pooling
Power
ReLU
Reshape
SoftMax
Tile

I also tried to programmatically find unsupported layers:

    plugin = IEPlugin(device=ARGS.device)  # MYRIAD
    supported_layers = plugin.get_supported_layers(net)
    not_supported_layers = [l for l in net.layers if l not in supported_layers]
    for l in sorted(not_supported_layers):
        print('{:<30}\t{}'.format(l, net.layers.type))

Some of them did show up, including Const and GEMM types (see below). However, both Const (?!) and GEMM are listed as "supported" by VPU from the documentation. Either the documentation is wrong, or the Python API is wrong, or something else.

0                             	Input
218/Output_0/Data__const      	Const
219/Output_0/Data__const      	Const
220/Output_0/Data__const      	Const
225/Output_0/Data__const      	Const
226/Output_0/Data__const      	Const
227/Output_0/Data__const      	Const
232/Output_0/Data__const      	Const
233/Output_0/Data__const      	Const
234/Output_0/Data__const      	Const
239/Output_0/Data__const      	Const
240/Output_0/Data__const      	Const
241/Output_0/Data__const      	Const
247/Output_0/Data_/copy_const 	Const
249/Output_0/Data_/copy_const 	Const
290                           	GEMM
298                           	GEMM
306                           	GEMM
315                           	GEMM
339                           	GEMM
395                           	GEMM
403                           	GEMM
411                           	GEMM
420                           	GEMM
444                           	GEMM
500                           	GEMM
508                           	GEMM
516                           	GEMM
525                           	GEMM
549                           	GEMM
605                           	GEMM
613                           	GEMM
621                           	GEMM
630                           	GEMM
654                           	GEMM

Is there a workaround or configuration that I can do? If not, when do we expect it to be fixed?

Thanks,
Alan

0 Kudos
Copy link
Shubha_R_Intel
Employee
‎08-28-2019 04:15 PM
800 Views

Dear Pan, Alan,

I think you may have found a bug.  I have PM'd you to get your version of the action recognition decoder script  where the target decoder device is NCS.

Please send it to me and I  will file a bug on your behalf. 

You can also attach your script as a *.zip to this forum ticket. 

Sorry for the trouble !

Shubha

 

0 Kudos
Copy link
Shubha_R_Intel
Employee
‎08-29-2019 01:32 PM
800 Views

Dear Pan, Alan,

Thank you for your cooperation (by sending me your Python script which demos the issue) ! I have filed a bug on your behalf.

Sincerely,

Shubha

0 Kudos
Copy link
Shubha_R_Intel
Employee
‎09-04-2019 12:53 PM
800 Views

Dear Pan, Alan

You'll be happy to know that this issue has been fixed in OpenVino R3.

Thanks for your patience !

Shubha

 

0 Kudos
Copy link
Reply

Community support is provided during standard business hours (Monday to Friday 7AM - 5PM PST). Other contact methods are available here.

Intel does not verify all solutions, including but not limited to any file transfers that may appear in this community. Accordingly, Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.

For more complete information about compiler optimizations, see our Optimization Notice.