Need an openvino python inference code that works with MYRIAD and raspberry pi4B

[Farhâd]

Have been looking over for an inference code in python that works with the stick 2 and can be used on raspberry pi.  I was able to finally with a lot of difficulty find the correct instruction page for the raspberry pi4 and stick2. Got all the sample models but want to use my own model that again with much difficulty I converted from pytorch to openvino ir.

The problem I have is that my codes require installing torch. I need an inference code in python that is torch free.

Please let me know.

[Farhâd]

Anyone? Any response would be appreciated.

[Hairul_Intel]

Hi Farhâd,

Thank you for reaching out to us.

OpenVINO™ provides various example codes and demo applications that supports Intel pre-trained models or public pre-trained models which are available from Open Model Zoo Demos.

According to the Dependencies for Python* Demos (requirements.txt file), Open Model Zoo demos does not require PyTorch dependencies in order to run the Python demos.

To clone the Open Model Zoo repository, you will need to git clone the specific branch of Open Model Zoo that matches your OpenVINO™ Toolkit package as follows:

git clone --depth 1 -b <openvino_version> https://github.com/openvinotoolkit/open_model_zoo.git

Regards,

Hairul

[Farhâd]

I think there is a misunderstanding. I have a model that I trained using my own data. The model is in pytorch. Need to run this model on a raspberry pi4b with MYRIAD stick 2. My problem is that the version of openvino I was able to install on my pi is 2021.3 as per this instruction page:

https://www.intel.com/content/www/us/en/support/articles/000057005/boards-and-kits.html

Th2 2021.3 requires openvino IR v10. But when I convert my pytorch model to openvino I get v11 which is incompatible with 2021.3 .

How can I convert my own home made model ( not a zoo model ) to be read by openvino 2021.3 so that I can run my model on my pi4b?

[Hairul_Intel]

Hi Farhâd,

You’ve mentioned that your Intermediate Representation (IR) model is in version IRv11, which can only be obtained via conversion using the latest OpenVINO™ Toolkit 2022 release.

Since your Raspberry Pi 4B is using OpenVINO™ 2021.3, I'd suggest you use the same OpenVINO™ version (OpenVINO™ 2021.3) when converting your model into IRv10. You can refer to Converting a PyTorch* Model for more information.

On another note, can you please provide the following information:

1. OpenVINO™ version that you used when converting the model.
2. Steps in converting your PyTorch model into IR format.
3. Model Optimizer scripts for converting your model into IR format.

Regards,

Hairul

[Farhâd]

Thank you.

I want to remind you I am trying to use openvino on raspberry pi4b. The only version that is allowed seems to be openvino 2021.3 that uses IRv10.  I have a model in pytroch but whenever I try to convert to openvino I get IRv11 that cannot  work with version 2021.3

The page you mentioned for conversion is for pytroch to ONNX.

Answers to you questions:

1. OpenVINO™ version that you used when converting the model.
   
   I use this export file from the latest YOLOv5 to convert pytorch to openvino. Have not been able to find any other code:
   https://github.com/ultralytics/yolov5/blob/master/export.py
   
   
2. Steps in converting your PyTorch model into IR format.
   
   On google colab:
   
   
   !git clone https://github.com/ultralytics/yolov5
   

   %cd yolov5

   !pip install -qr requirements.txt

   !pip install onnx>=1.1.2

   !export PYTHONPATH="$PWD" && python export.py --weights <my custom pytorch model> --img 512 --include openvino

   >>result:

   OpenVINO runtime found in: 	/usr/local/lib/python3.7/dist-packages/openvino
   OpenVINO runtime version: 	2022.2.0-7713-af16ea1d79a-releases/2022/2
   Model Optimizer version: 	2022.2.0-7713-af16ea1d79a-releases/2022/2
   [ SUCCESS ] Generated IR version 11 model.
   [ SUCCESS ] XML file: /content/yolov5/best_10-28-22_epoch=100_openvino_model/best_10-28-22_epoch=100.xml
   [ SUCCESS ] BIN file: /content/yolov5/best_10-28-22_epoch=100_openvino_model/best_10-28-22_epoch=100.bin
   [ SUCCESS ] Total execution time: 1.13 seconds. 
   [ SUCCESS ] Memory consumed: 150 MB. 
   [ INFO ] The model was converted to IR v11, the latest model format that corresponds to the source DL framework input/output format. While IR v11 is backwards compatible with OpenVINO Inference Engine API v1.0, please use API v2.0 (as of 2022.1) to take advantage of the latest improvements in IR v11.
   Find more information about API v2.0 and IR v11 at https://docs.openvino.ai
   OpenVINO: export success  4.0s, saved as best_10-28-22_epoch=100_openvino_model/ (27.3 MB)
   
   Export complete (5.8s)
   Results saved to /content/yolov5
   Detect:          python detect.py --weights best_10-28-22_epoch=100_openvino_model/ 
   Validate:        python val.py --weights best_10-28-22_epoch=100_openvino_model/ 
   PyTorch Hub:     model = torch.hub.load('ultralytics/yolov5', 'custom', 'best_10-28-22_epoch=100_openvino_model/')  
   Visualize:       https://netron.app
   

3. Model Optimizer scripts for converting your model into IR format.
   The MO command is in the export.py supplied with YOLOv5 ( link above) lines 197 and 198:
   

   cmd = f"mo --input_model {file.with_suffix('.onnx')} --output_dir {f} --data_type {'FP16' if half else 'FP32'}"

   subprocess.run(cmd.split(), check=True, env=os.environ) # export
   
   Hope this helps. I appreciate your help
   

[Hairul_Intel · Accepted solution]

Hi Farhâd,

Thank you for sharing your detailed information.

For your information, I've encountered errors and was unable to convert a yolov5s model into IRv10 using OpenVINO™ 2021.3 in Google Colab. This error is due to incompatible dependencies when converting the model. Thus, I'd suggest you to keep using your successfully converted IRv11 model instead.

As per my suggestion, you might want to build OpenVINO™ 2022.1 from source on your Raspberry Pi instead as it will allow you to use OpenVINO™ API 2.0 and also read IRv11 model.

Regards,

Hairul 

[Hairul_Intel]

Hi Farhâd,

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question.

Regards,

Hairul