How to use HETERO and MULTI device in YOLO demo

[WU__Tianwen]

https://docs.openvinotoolkit.org/latest/omz_demos_python_demos_object_detection_demo_yolov3_async_README.html

this document only shows“python3 object_detection_demo_yolov3_async.py -i <path_to_video>/inputVideo.mp4 -m <path_to_model>/yolo_v3.xml -d GPU”

but i don't know how to use CPU+GPU and how to set parameters of "-nstreams -nthreads -nireq -d "to fully implement these functions  

Can you give me an example if I want to use CPU and GPU

args.add_argument("-d", "--device",
                  help="Optional. Specify the target device to infer on; CPU, GPU, FPGA, HDDL or MYRIAD is"
                       " acceptable. The sample will look for a suitable plugin for device specified. "
                       "Default value is CPU", default="CPU", type=str)

args.add_argument("-nireq", "--num_infer_requests", help="Optional. Number of infer requests",
 default=1, type=int)
args.add_argument("-nstreams", "--num_streams",
 help="Optional. Number of streams to use for inference on the CPU or/and GPU in throughput mode "
 "(for HETERO and MULTI device cases use format <device1>:<nstreams1>,<device2>:<nstreams2> "
 "or just <nstreams>)",
 default="", type=str)
args.add_argument("-nthreads", "--number_threads",
 help="Optional. Number of threads to use for inference on CPU (including HETERO cases)",
 default=None, type=int)

[Iffa_Intel]

Greetings,

In order to use GPU you can use command like this:

python3 object_detection_demo_yolov3_async.py -i <path_to_video>/inputVideo.mp4 -m <path_to_model>/yolo_v3.xml -d GPU

Same concept for CPU:

python3 object_detection_demo_yolov3_async.py -i <path_to_video>/inputVideo.mp4 -m <path_to_model>/yolo_v3.xml -d CPU

To run the demo, you can use public or pre-trained models. You can download the pre-trained models with the OpenVINO Model Downloader or from https://download.01.org/opencv/.

Here is the full official documentation:

https://docs.openvinotoolkit.org/latest/omz_demos_python_demos_object_detection_demo_yolov3_async_README.html

NOTE: Before running the demo with a trained model, make sure the model is converted to the Inference Engine format (*.xml + *.bin) using the Model Optimizer tool.

Sincerely,

Iffa

[WU__Tianwen]

Hello!

I know -d CPU to use cpu,and -d GPU to use GPU.But my question is how to use HETERO and MULTI device.  How to set   -nstreams ,-nireq ,-nthreads.The guide book didn't show how to use these new parameters

args.add_argument("-nireq", "--num_infer_requests", help="Optional. Number of infer requests",
 default=1, type=int)
args.add_argument("-nstreams", "--num_streams",
 help="Optional. Number of streams to use for inference on the CPU or/and GPU in throughput mode "
 "(for HETERO and MULTI device cases use format <device1>:<nstreams1>,<device2>:<nstreams2> "
 "or just <nstreams>)",
 default="", type=str)
args.add_argument("-nthreads", "--number_threads",
 help="Optional. Number of threads to use for inference on CPU (including HETERO cases)",
 default=None, type=int)

[Iffa_Intel · Accepted solution]

Here is how to use that in Hetero:

./object_detection_sample_ssd -m <path_to_model>/ModelSSD.xml -i <path_to_pictures>/picture.jpg -d HETERO:FPGA,CPU

you can refer to this Hetero documentation: https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_supported_plugins_HETERO.html

and this is how you can use wutl Multi:

./benchmark_app –d MULTI:CPU,GPU –m <model> -i <input> -niter 1000

you can refer to this Multi documentation:

https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_supported_plugins_MULTI.html

Sincerely,

Iffa

[Iffa_Intel]

Greetings,

Intel will no longer monitor this thread since we have provided a solution. If you need any additional information from Intel, please submit a new question

Sincerely,

Iffa