Intel® Distribution of OpenVINO™ Toolkit
Community support and discussions about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all things computer vision-related on Intel® platforms.
5924 Discussions

What backend does OpenVino inference Benchmark use?

Joon
Beginner
526 Views

Hello, 

I am trying to run the openvino inference becnhmark sample. However, I was wondering what back-end (runtime) does openvino use? To be specific, some inference benchmark uses tensorflow, onnxruntime, etc. However, during use the openvino inference benchmark I was not able to set up these back-end.
Therefore, I was wondering if openvino inference benchmark uses its own runtime. 

Also, what I understand is that size of memory definitely can affect the results of throughput. However, even if I increase the size of memory 8GB to 16GB, the overall throughput are similar. Is it because I run the benchmark differently? or the size of memory does not really affect the results?

I am new to the openvino tools. Please correct me if I misunderstand something!
Thank you so much for the time and consideration

0 Kudos
1 Solution
JesusE_Intel
Moderator
481 Views

Hi Joon,


The OpenVINO toolkit uses the Inference Engine to read, load and inference the model (Intermediate Representation) on the target device. You can find additional information on the Inference Engine Documentation. The benchmark app is measuring the latency for each executed infer request, please see the How It Works section in the app documentation.


Regards,

Jesus


View solution in original post

3 Replies
JesusE_Intel
Moderator
482 Views

Hi Joon,


The OpenVINO toolkit uses the Inference Engine to read, load and inference the model (Intermediate Representation) on the target device. You can find additional information on the Inference Engine Documentation. The benchmark app is measuring the latency for each executed infer request, please see the How It Works section in the app documentation.


Regards,

Jesus


JesusE_Intel
Moderator
472 Views

Hi Joon,


Just following up to see if you have any additional questions after my last response.


Regards,

Jesus


JesusE_Intel
Moderator
465 Views

Intel will no longer monitor this thread since we have provided a solution. If you need any additional information from Intel, please submit a new question. 


Reply