Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Joon
Beginner
309 Views

What backend does OpenVino inference Benchmark use?

Jump to solution

Hello, 

I am trying to run the openvino inference becnhmark sample. However, I was wondering what back-end (runtime) does openvino use? To be specific, some inference benchmark uses tensorflow, onnxruntime, etc. However, during use the openvino inference benchmark I was not able to set up these back-end.
Therefore, I was wondering if openvino inference benchmark uses its own runtime. 

Also, what I understand is that size of memory definitely can affect the results of throughput. However, even if I increase the size of memory 8GB to 16GB, the overall throughput are similar. Is it because I run the benchmark differently? or the size of memory does not really affect the results?

I am new to the openvino tools. Please correct me if I misunderstand something!
Thank you so much for the time and consideration

0 Kudos
1 Solution
JesusE_Intel
Moderator
264 Views

Hi Joon,


The OpenVINO toolkit uses the Inference Engine to read, load and inference the model (Intermediate Representation) on the target device. You can find additional information on the Inference Engine Documentation. The benchmark app is measuring the latency for each executed infer request, please see the How It Works section in the app documentation.


Regards,

Jesus


View solution in original post

3 Replies
JesusE_Intel
Moderator
265 Views

Hi Joon,


The OpenVINO toolkit uses the Inference Engine to read, load and inference the model (Intermediate Representation) on the target device. You can find additional information on the Inference Engine Documentation. The benchmark app is measuring the latency for each executed infer request, please see the How It Works section in the app documentation.


Regards,

Jesus


View solution in original post

JesusE_Intel
Moderator
255 Views

Hi Joon,


Just following up to see if you have any additional questions after my last response.


Regards,

Jesus


JesusE_Intel
Moderator
248 Views

Intel will no longer monitor this thread since we have provided a solution. If you need any additional information from Intel, please submit a new question. 


Reply