Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6386 Discussions

How to use HETERO and MULTI device in YOLO demo

WU__Tianwen
New Contributor I
1,155 Views

https://docs.openvinotoolkit.org/latest/omz_demos_python_demos_object_detection_demo_yolov3_async_README.html

this document only shows“python3 object_detection_demo_yolov3_async.py -i <path_to_video>/inputVideo.mp4 -m <path_to_model>/yolo_v3.xml -d GPU”

but i don't know how to use CPU+GPU and how to set parameters of "-nstreams -nthreads -nireq -d "to fully implement these functions  

Can you give me an example if I want to use CPU and GPU

args.add_argument("-d", "--device",
help="Optional. Specify the target device to infer on; CPU, GPU, FPGA, HDDL or MYRIAD is"
" acceptable. The sample will look for a suitable plugin for device specified. "
"Default value is CPU", default="CPU", type=str)

args.add_argument("-nireq", "--num_infer_requests", help="Optional. Number of infer requests",
default=1, type=int)
args.add_argument("-nstreams", "--num_streams",
help="Optional. Number of streams to use for inference on the CPU or/and GPU in throughput mode "
"(for HETERO and MULTI device cases use format <device1>:<nstreams1>,<device2>:<nstreams2> "
"or just <nstreams>)",
default="", type=str)
args.add_argument("-nthreads", "--number_threads",
help="Optional. Number of threads to use for inference on CPU (including HETERO cases)",
default=None, type=int)

 

0 Kudos
1 Solution
Iffa_Intel
Moderator
1,113 Views

Here is how to use that in Hetero:

./object_detection_sample_ssd -m <path_to_model>/ModelSSD.xml -i <path_to_pictures>/picture.jpg -d HETERO:FPGA,CPU


you can refer to this Hetero documentation: https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_supported_plugins_HETERO.html


and this is how you can use wutl Multi:

./benchmark_app –d MULTI:CPU,GPU –m <model> -i <input> -niter 1000


you can refer to this Multi documentation:

https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_supported_plugins_MULTI.html


Sincerely,

Iffa


View solution in original post

0 Kudos
4 Replies
Iffa_Intel
Moderator
1,134 Views

Greetings,


In order to use GPU you can use command like this:

python3 object_detection_demo_yolov3_async.py -i <path_to_video>/inputVideo.mp4 -m <path_to_model>/yolo_v3.xml -d GPU


Same concept for CPU:

python3 object_detection_demo_yolov3_async.py -i <path_to_video>/inputVideo.mp4 -m <path_to_model>/yolo_v3.xml -d CPU


To run the demo, you can use public or pre-trained models. You can download the pre-trained models with the OpenVINO Model Downloader or from https://download.01.org/opencv/.


Here is the full official documentation:

https://docs.openvinotoolkit.org/latest/omz_demos_python_demos_object_detection_demo_yolov3_async_README.html


NOTE: Before running the demo with a trained model, make sure the model is converted to the Inference Engine format (*.xml + *.bin) using the Model Optimizer tool.



Sincerely,

Iffa


0 Kudos
WU__Tianwen
New Contributor I
1,126 Views

Hello!

I know -d CPU to use cpu,and -d GPU to use GPU.But my question is how to use HETERO and MULTI device.  How to set   -nstreams ,-nireq ,-nthreads.The guide book didn't show how to use these new parameters

args.add_argument("-nireq", "--num_infer_requests", help="Optional. Number of infer requests",
default=1, type=int)
args.add_argument("-nstreams", "--num_streams",
help="Optional. Number of streams to use for inference on the CPU or/and GPU in throughput mode "
"(for HETERO and MULTI device cases use format <device1>:<nstreams1>,<device2>:<nstreams2> "
"or just <nstreams>)",
default="", type=str)
args.add_argument("-nthreads", "--number_threads",
help="Optional. Number of threads to use for inference on CPU (including HETERO cases)",
default=None, type=int)

 

0 Kudos
Iffa_Intel
Moderator
1,114 Views

Here is how to use that in Hetero:

./object_detection_sample_ssd -m <path_to_model>/ModelSSD.xml -i <path_to_pictures>/picture.jpg -d HETERO:FPGA,CPU


you can refer to this Hetero documentation: https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_supported_plugins_HETERO.html


and this is how you can use wutl Multi:

./benchmark_app –d MULTI:CPU,GPU –m <model> -i <input> -niter 1000


you can refer to this Multi documentation:

https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_supported_plugins_MULTI.html


Sincerely,

Iffa


0 Kudos
Iffa_Intel
Moderator
1,109 Views

Greetings,


Intel will no longer monitor this thread since we have provided a solution. If you need any additional information from Intel, please submit a new question


Sincerely,

Iffa


0 Kudos
Reply