topic Re:How to use HETERO and MULTI device in YOLO demo in Intel® Distribution of OpenVINO™ Toolkit

How to use HETERO and MULTI device in YOLO demo

WU__Tianwen — Tue, 28 Jul 2020 15:56:40 GMT

https://docs.openvinotoolkit.org/latest/omz_demos_python_demos_object_detection_demo_yolov3_async_README.html

this document only shows“python3 object_detection_demo_yolov3_async.py -i <path_to_video>/inputVideo.mp4 -m <path_to_model>/yolo_v3.xml -d GPU”

but i don't know how to use CPU+GPU and how to set parameters of "-nstreams -nthreads -nireq -d "to fully implement these functions

Can you give me an example if I want to use CPU and GPU

args.add_argument("-d", "--device",
                  help="Optional. Specify the target device to infer on; CPU, GPU, FPGA, HDDL or MYRIAD is"
                       " acceptable. The sample will look for a suitable plugin for device specified. "
                       "Default value is CPU", default="CPU", type=str)


args.add_argument("-nireq", "--num_infer_requests", help="Optional. Number of infer requests",
 default=1, type=int)
args.add_argument("-nstreams", "--num_streams",
 help="Optional. Number of streams to use for inference on the CPU or/and GPU in throughput mode "
 "(for HETERO and MULTI device cases use format <device1>:<nstreams1>,<device2>:<nstreams2> "
 "or just <nstreams>)",
 default="", type=str)
args.add_argument("-nthreads", "--number_threads",
 help="Optional. Number of threads to use for inference on CPU (including HETERO cases)",
 default=None, type=int)

Re:How to use HETERO and MULTI device in YOLO demo

Iffa_Intel — Thu, 30 Jul 2020 05:49:52 GMT

Greetings,

In order to use GPU you can use command like this:

python3 object_detection_demo_yolov3_async.py -i <path_to_video>/inputVideo.mp4 -m <path_to_model>/yolo_v3.xml -d GPU

Same concept for CPU:

python3 object_detection_demo_yolov3_async.py -i <path_to_video>/inputVideo.mp4 -m <path_to_model>/yolo_v3.xml -d CPU

To run the demo, you can use public or pre-trained models. You can download the pre-trained models with the OpenVINO Model Downloader or from https://download.01.org/opencv/.

Here is the full official documentation:

https://docs.openvinotoolkit.org/latest/omz_demos_python_demos_object_detection_demo_yolov3_async_README.html

NOTE: Before running the demo with a trained model, make sure the model is converted to the Inference Engine format (*.xml + *.bin) using the Model Optimizer tool.

Sincerely,

Iffa

Re: Re:How to use HETERO and MULTI device in YOLO demo

WU__Tianwen — Thu, 30 Jul 2020 08:35:30 GMT

Hello!

I know -d CPU to use cpu,and -d GPU to use GPU.But my question is how to use HETERO and MULTI device. How to set -nstreams ,-nireq ,-nthreads.The guide book didn't show how to use these new parameters

args.add_argument("-nireq", "--num_infer_requests", help="Optional. Number of infer requests",
 default=1, type=int)
args.add_argument("-nstreams", "--num_streams",
 help="Optional. Number of streams to use for inference on the CPU or/and GPU in throughput mode "
 "(for HETERO and MULTI device cases use format <device1>:<nstreams1>,<device2>:<nstreams2> "
 "or just <nstreams>)",
 default="", type=str)
args.add_argument("-nthreads", "--number_threads",
 help="Optional. Number of threads to use for inference on CPU (including HETERO cases)",
 default=None, type=int)

Re:How to use HETERO and MULTI device in YOLO demo

Iffa_Intel — Tue, 04 Aug 2020 07:54:56 GMT

Here is how to use that in Hetero:

./object_detection_sample_ssd -m <path_to_model>/ModelSSD.xml -i <path_to_pictures>/picture.jpg -d HETERO:FPGA,CPU

you can refer to this Hetero documentation: https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_supported_plugins_HETERO.html

and this is how you can use wutl Multi:

./benchmark_app –d MULTI:CPU,GPU –m <model> -i <input> -niter 1000

you can refer to this Multi documentation:

https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_supported_plugins_MULTI.html

Sincerely,

Iffa

Re:How to use HETERO and MULTI device in YOLO demo

Iffa_Intel — Thu, 06 Aug 2020 06:23:49 GMT

Greetings,

Intel will no longer monitor this thread since we have provided a solution. If you need any additional information from Intel, please submit a new question

Sincerely,

Iffa