Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6528 Discussions

Clarification on Inference Performance with Intel DL Streamer

tx-y-hiraka
Novice
1,434 Views

Summary:

I have  a question regarding inference performance using Intel DL Streamer and would appreciate some insights.

When performing inference on two CPUs with different FLOPS, the results were contrary to expectations. The CPU with the lower FLOPS value completed the inference faster, while the one with the higher FLOPS value was unexpectedly slower.

What could be the possible factors for this?

 

Details:

I have a custom object detection model that has been fine-tuned from Yolov3. I benchmarked this model using the benchmark tool available inside the intel/dlstreamer:dgpu-dpcpp-devel container.

 

 

 

./benchmark_app -m custom_yolov3.xml

 

 

 

The two different CPUs are:

a. Xeon Bronze 3106

b. Atom x6425E

 

Unfortunately, I am not proficient in calculating FLOPS, so I have sourced the FLOPS values from external websites.

 

a. Xeon Bronze 3106

https://www.hpc.co.jp/product/cpu_intel_xeon-scalable-processors/ 

I apologize for the Japanese website, but it appears to be written as 166.4 GFLOPS.

 

b. Atom x6425E

 https://www.cpu-monkey.com/en/cpu-intel_atom_x6425e 

Although it is the FLOPS with iGPU, it seems to be listed as 400 GFLOPS.

 

My hypotheses include:

  1. Intel DLStreamer may not always perform inference using iGPU.
  2. The Intel Chipsets Driver, which should normally be integrated into Linux, may not be functioning correctly.
  3. The Atom CPU may lack certain CPU instructions like AVX, which could be causing the issue.

I would like to uncover the truth, so if anyone has any knowledge on this matter, please share your insights.

 

By the way, the operating system, main memory, and storage conditions are almost identical between the two computers.

0 Kudos
1 Solution
Zulkifli_Intel
Moderator
1,281 Views

Hi x-y-hiraka,

 

I'm glad that you find my reply helpful.

 

Intel Atom® x6425E Processor uses Intel® UHD Graphics for 10th Gen Intel® Processors as integrated GPU (refer here).


Based on Hardware Requirements, it states that Intel® DL Streamer releases are validated on the Intel® Data Center GPU Flex Series. But the full list of potentially supported platforms can be found on System Requirements, and Intel® UHD Graphics is listed as one of the supported platforms. Hope this information helps.

 

 

Regards,

Zulkifli 


View solution in original post

4 Replies
Zulkifli_Intel
Moderator
1,365 Views

Hi tx-y-hiraka,

 

Thank you for reaching out to us.


Different DL Streamer inference elements in the pipeline can run on different processing units such as CPU, GPU, and VPU. Each element has a property named 'device' which is used for specifying the target device/ processing unit for that element. 


When comparing the Intel Xeon Bronze 3106 and the Intel Atom x6425E for inferencing tasks, it's essential to consider several factors beyond raw performance metrics like GFLOPS. The Xeon Bronze 3106 is designed for a range of server and data center workloads, including virtualization, cloud computing, and enterprise applications. It typically offers higher performance and more features compared to the Atom x6425E which is designed for low-DMIP computing and low-speed I/Os for IoT applications. Therefore the inference speed is most likely to be faster for Xeon Bronze 3106 compared to Atom x6425E. 

 

 

Regards,

Zul


tx-y-hiraka
Novice
1,324 Views

@Zulkifli_Intel 

I am pleased to receive your response. Thank you very much indeed. It seems that factors other than FLOPS, such as I/O, do indeed contribute to differences in inference performance. Your post has been very informative.

On the other hand, I would like to do some more research to find out whether the Atom's iGPU can be used or not.

GPU device selection — Intel® Deep Learning Streamer (Intel® DL Streamer) documentation

 

Regards,

Yuya

0 Kudos
Zulkifli_Intel
Moderator
1,282 Views

Hi x-y-hiraka,

 

I'm glad that you find my reply helpful.

 

Intel Atom® x6425E Processor uses Intel® UHD Graphics for 10th Gen Intel® Processors as integrated GPU (refer here).


Based on Hardware Requirements, it states that Intel® DL Streamer releases are validated on the Intel® Data Center GPU Flex Series. But the full list of potentially supported platforms can be found on System Requirements, and Intel® UHD Graphics is listed as one of the supported platforms. Hope this information helps.

 

 

Regards,

Zulkifli 


Zulkifli_Intel
Moderator
1,236 Views

Hi x-y-hiraka,

 

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question.

 


Regards,

Zulkifli


0 Kudos
Reply