A todo list to run pre-trained "ASL recognition demo" on RPI4b with NCS2 & WebCam

[pang__pih]

Hi,

Could a smart person, please list what I need to do to run this asl-recog-0004-fp16 demo.?

Is all I need two IR .xml files(plus matching .bin files) detector and recognition xml?

https://docs.openvinotoolkit.org/latest/_models_intel_age_gender_recognition_retail_0013_description_age_gender_recognition_retail_0013.html

Openvino toolkit 2020.1:
asl-recognition-0003.xml

asl-recognition-0003.bin

In /opt/intel/openvino/deployment_tools/open_model_zoo/demos/python_demos/asl_recognition_demo/asl_recognition_demo.py

    args.add_argument('-m_a', '--action_model',
                      help='Required. Path to an .xml file with a trained asl recognition model.',  example : asl-recognition-0003.xml
                      required=True, type=str)
    args.add_argument('-m_d', '--detection_model',
                      help='Required. Path to an .xml file with a trained person detector model.',  example : person-detection-asl-0001.xml
                      required=True, type=str)
    args.add_argument('-i', '--input',
                      help='Required. Path to a video file or a device node of a web-camera.', example : -i 0 or -i cam ??
                      required=True, type=str)

https://download.01.org/opencv/2020/openvinotoolkit/2020.1/open_model_zoo/models_bin/1/asl-recognition-0003/FP32/

https://download.01.org/opencv/2020/openvinotoolkit/2020.1/open_model_zoo/models_bin/1/person-detection-asl-0001/FP32/

What is the device node of a web-camera in RPi4B?

Another question if ASL demo is too complicated..

What is the first best(easiest first pre-trained model to demo on Rpi4b ,Webcam and NCS2?

Can Rpi4B be used by all Open_Model_Zoo models and all numbertypes: FP16, FP16+Int8, FP32?

On a RPi4B with Raspian(Buster) how can I see the device node name of an attached webcam?

Thanks, appreciate any feedback!

Pih Lung

[SIRIGIRI_V_Intel]

Hi Pih,

To run the asl-recognition-demo on Raspbery 4B, Install the OpenVINO toolkit for Raspberry pi and follow the Readme to run the demo.

Connect the camera and use the below command to get the list of devices available:

ls /dev/video*

For example, if the output of above command is /dev/video0, then the camera id is 0.

Use FP16 models to get the good accuracy on Raspberry pi.

Hope this helps.

Regards,

Ram prasad

[David_C_Intel]

Hi Pih,

Thanks for reaching out.

1. Regarding with the asl_recognition_demo, you have FP32 precision for one of the pre trained models used, so it cannot be executed in Rpi, as you need the Myriad plugin and FP16 precision IR files. 
2. The device node of a web camera can be found under /dev/video#, which "#" points to each camera device, if you have connected only one, it usually is "video0". In the asl_recognition_demo in particular, you use the input -i 0, but most of the demos use the -i cam input, you can read each sample and demo documentation in the Resources tab.
3. If you want to use Rpi + cam + NCS2, you can check the Python/C++ object_detection_demo_ssd_async  or object_detection_demo_yolov3_async demos.
4. The Rpi only supports the Myriad plugin, so only those models supported by the Myriad plugin work. Same applies to data types, you can check this supported configurations documentation for more information. Additionally, you can check the networks supported by the Myriad plugin and the demos supported for each pre-trained model.

Best regards,

David C.

[pang__pih]

Hi Ram and David,

Thanks for your replies, they helped me progress!

But I hit an obstacle. I ran this command from my Rpi4b with NCS2 and webcamera.

python3 /opt/intel/openvino/deployment_tools/open_model_zoo/demos/python_demos/asl_recognition_demo/asl_recognition_demo.py -i cam -m_a ./asl-recognition-0003.xml -m_d ./person-detection-asl-0001.xml -d MYRIAD -c ./classes.json

pi@raspberrypi:~/Desktop/ASL_recognition_demo $ ./runASLdemo.sh 
Traceback (most recent call last):
  File "/opt/intel/openvino/deployment_tools/open_model_zoo/demos/python_demos/asl_recognition_demo/asl_recognition_demo.py", line 184, in <module>
    sys.exit(main() or 0)
  File "/opt/intel/openvino/deployment_tools/open_model_zoo/demos/python_demos/asl_recognition_demo/asl_recognition_demo.py", line 104, in main
    num_requests=2, output_shape=DETECTOR_OUTPUT_SHAPE)
  File "/opt/intel/openvino/deployment_tools/open_model_zoo/demos/python_demos/asl_recognition_demo/asl_recognition_demo/person_detector.py", line 29, in __init__
    super().__init__(model_path, device, ie_core, num_requests, output_shape)
  File "/opt/intel/openvino/deployment_tools/open_model_zoo/demos/python_demos/asl_recognition_demo/asl_recognition_demo/common.py", line 39, in __init__
    self.net = IENetwork(model=model_xml, weights=model_bin)
  File "ie_api.pyx", line 980, in openvino.inference_engine.ie_api.IENetwork.__cinit__
RuntimeError: Check 'axis < static_cast<size_t>(input_rank)' failed at /teamcity/work/scoring_engine_build/releases_2020_1/ngraph/src/ngraph/op/gather.cpp:140:
While validating node 'Gather[Gather_2865](patternLabel_2861: float{10,20,30}, patternLabel_2862: int64_t{5}, patternLabel_2864: int64_t{1}) -> (??)':
The axis must => 0 and <= input_rank (axis: 4294967295).

pi@raspberrypi:~/Desktop/ASL_recognition_demo $ 
 

I few questions:

I am using openvino 2020.1, and in this version there is only FP32 version of person-detection-asl-0001.xml, but in 2020.2 there is also an FP16 of the file. So I used the 2020.2 FP16 version. Is that a problem?

The errror:  "The axis must => 0 and <= input_rank (axis: 4294967295)." indicate that the IR need to be downgraded to Version 7 right. Is it possible that you can generate IRv7 files and make them available to me? Or can you provide me with the trained YOLOv3 model so I can run the mo.py with --generate_deprecated_IR_v7 param myself? 

Please, advise!

Cheers

Pih Lung

[SIRIGIRI_V_Intel]

Are you using the model from this link?

It is not recommended to use models from here as there could be pre-release contents that are planned to be included into OpenVINO 2020.2 release, which is not released yet.

Could you try using the person-detection-retail-0013 instead of person-detection-asl-0001 ?

Regards,

Ram prasad

[pang__pih]

Hi Ram,

Ran localRunASLdemo.sh that consist of this line:

python3 ./asl_recognition_demo.py -i cam -m_a ./asl-recognition-0003.xml -m_d ./person-detection-retail-0013.xml -d MYRIAD -c ./classes.json

person-detection-retail-0013.xml is the FP16 version downloaded from 2020.1

gives the same output:

pi@raspberrypi:~/Desktop/ASL_recognition_demo $ ./localRunASLdemo.sh 
Traceback (most recent call last):
  File "./asl_recognition_demo.py", line 184, in <module>
    sys.exit(main() or 0)
  File "./asl_recognition_demo.py", line 104, in main
    num_requests=2, output_shape=DETECTOR_OUTPUT_SHAPE)
  File "/home/pi/Desktop/ASL_recognition_demo/asl_recognition_demo/person_detector.py", line 29, in __init__
    super().__init__(model_path, device, ie_core, num_requests, output_shape)
  File "/home/pi/Desktop/ASL_recognition_demo/asl_recognition_demo/common.py", line 39, in __init__
    self.net = IENetwork(model=model_xml, weights=model_bin)
  File "ie_api.pyx", line 980, in openvino.inference_engine.ie_api.IENetwork.__cinit__
RuntimeError: Check 'axis < static_cast<size_t>(input_rank)' failed at /teamcity/work/scoring_engine_build/releases_2020_1/ngraph/src/ngraph/op/gather.cpp:140:
While validating node 'Gather[Gather_7680](patternLabel_7676: float{10,20,30}, patternLabel_7677: int64_t{5}, patternLabel_7679: int64_t{1}) -> (??)':
The axis must => 0 and <= input_rank (axis: 4294967295).

pi@raspberrypi:~/Desktop/ASL_recognition_demo $ 
 

Regards,

Pih Lung

[pang__pih]

3. If you want to use Rpi + cam + NCS2, you can check the Python/C++ object_detection_demo_ssd_async  or object_detection_demo_yolov3_async demos.

Hi David,

I am able to run object_detection_demo_yolov3_async demo using Coco model (with assistance from Jaivin see https://software.intel.com/comment/1956034)

But all along I wanted to do ASL recognition, and is that possible with "object_detection_demo_yolov3_async" demo?

It only takes one model -m as input, the ASL demo takes two -m_a and -m_d... if it can please provide the call with input params.

Thanks

Pih Lung

[SIRIGIRI_V_Intel]

Hi Pang,

Possible solutions for this issue are mentioned in this thread. Please refer this and let us know if this helps.

Regards,

Ram prasad

[David_C_Intel]

Hi Pih,

Thank you for your patience. Please check the asl_recognition_demo, we tested it with the OpenVINO™ toolkit 2020.1 version and it worked successfully. As previously mentioned this demo does not work on Rpi, as one of its models use FP32 precision, you have to run it in a Linux/Windows/Mac OS based system. You can find the files for this demo in the following directory:

<OpenVINO_INSTALL_DIR>/deployment_tools/open_model_zoo/demos/python_demos/asl_recognition_demo/

After downloading the .xml files with their .bin files run the following command:

python3 <ABS_PATH>/asl_recognition_demo.py  -m_a <ABS_PATH>/asl-recognition-0003.xml  -m_d <ABS_PATH>/person-detection-asl-0001.xml  -i 0  -c <ABS_PATH>/classes.json

Best Regards,

David

[pang__pih]

Hi David,

Thanks for your reply and precise command of how to call the asl example.

In toolkit 2020.2 there is a FP16 version of person-detection-asl-0001.xml that in 2020.1 only had a FP32 precision.

Does it mean that the asl model can run on Raspberry Pi 4b, when 2020.2 releases a l_openvino_toolkit_runtime_raspian_p_2020.2 image?

And when can we expect that release?

Have a good day.

[ROCK]

hello,i have countered the same problem.

can the FP16 model use by raspberry pi 4?

this is the question i get：

https://community.intel.com/t5/Intel-Distribution-of-OpenVINO/Device-with-quot-CPU-quot-name-is-not-registered-in-the/m-p/1241148/highlight/true#M22108

[SIRIGIRI_V_Intel]

OpenVINO 2020.2 is released. Please try to use the latest OpenVINO.

Regards,

Ram prasad

[engineer0204]

now OpenVINO 2021.1 is released.

but the asl_recognition_demo this model is still not Supported on MyriadX