inference performance is not reached in Yocto OS

[BSung8]

Dear sir,

I tested YoloV3 (416x416 INT8), inference performance just only 47.3fps in Yocto.

As the same HW with ubuntu20.04, the performance would be 58fps.

But this happened only I7-1185G7E. 

I5-1145G7E, I3-1115G4E, Celeron 6305E inference performance in yocto would be same as ubuntu20.04.

my test environment:

TGL yocto version: release-74_tgl_u_mr3

openvino version: 2021.4

Any comments? 

Thanks!

[IntelSupport]

Hi BSung8,

Thanks for reporting this to us.

We will have a look at this and check with our development team for further clarification.

Meanwhile, please be noted that the Yocto image with OpenVINO toolkit was last validated and tested for

Yocto OpenVINO 2020.4 release.

Regards,

Aznie

[BSung8]

Hi Aznie,

Thanks for your reply.

My test YoloV3 model: https://drive.google.com/file/d/1MiB5W-khMNRCSw6sB5tM5MvvUQWyLiJe/view?usp=sharing

test application: object_detection_demo cpp

inference device : GPU

Thanks!!

[BSung8]

Hi Aznie,

Where I can download OpenVINO 2020.4 release. BB file and patch ??

BR

[IntelSupport]

Hi BSung8,

I noticed you are using the TGL Yocto version: release-74_tgl_u_mr3, can you share the source of this e.g. installer or GitHub repo that you used?

Regards,

Aznie

[BSung8]

repo init -u https://github.com/intel/iotg-yocto-ese-manifest-b refs/tags/release-74_tgl_u_mr3 -g all

[IntelSupport]

Hi BSung8,

Thanks for sharing the repo.

Can you share how you port the optimized model to run the inference on both platforms? Meanwhile, I would suggest you perform the model optimizer within the same environment prior to inferencing (e.g Model Optimizer in Yocto, IE in Yocto).

Regards,

Aznie

[BSung8]

My test YoloV3 model: https://drive.google.com/file/d/1MiB5W-khMNRCSw6sB5tM5MvvUQWyLiJe/view?usp=sharing

I will try MO in Yocto.

[Munesh_Intel]

Hi BSung8,

Thank you for your patience, and our apologies for the delay in replying to you.

For your query regarding inference performance deterioration in Yocto OS (as compared to Ubuntu OS), the source of the Yocto BSP image that you used was not validated by OpenVINO. As such, we are unable to replicate nor confirm the inference performance.

The instructions for creating a Yocto image with OpenVINO™ toolkit are available in Create a Yocto Image with OpenVINO™ toolkit. We would suggest you create a Yocto image by following the steps mentioned in the link above to validate further the inference performance for GPU using Yocto OS.

Thank you.

Regards,

Munesh

[BSung8]

Hi Munesh,

I follow Create a Yocto Image with OpenVINO™ toolkit.  to build yocto image. 

After build up the image, openVINO version is 2021.4.

I can't get validated version 2020.4 BB file in meta-intel.

[Munesh_Intel]

Hi BSung8,

Yes, you’re right. The version of OpenVINO in the master branch will always reflect the latest released OpenVINO version. Right now, it’s version 2021.4.2.

We mentioned the 2020.4 version, as it was the last version where the build instructions were validated by our Engineering team. The build instructions for the subsequent releases were not validated.

Do share with us your validation results for the benefit of our Community.

Regards,

Munesh

[BSung8]

Hi Munesh,

I tested YoloV3 (416x416 INT8), inference performance just only 47.3fps in Yocto.

As the same HW with ubuntu20.04, the performance would be 58fps.

But this happened only I7-1185G7E. 

I5-1145G7E, I3-1115G4E, Celeron 6305E inference performance in yocto would be same as ubuntu20.04.

my test environment:

TGL yocto version: release-74_tgl_u_mr3

openvino version: 2021.4

My test YoloV3 model: https://drive.google.com/file/d/1MiB5W-khMNRCSw6sB5tM5MvvUQWyLiJe/view?usp=sharing

test application: object_detection_demo cpp

inference device : GPU

Thanks!!

[Munesh_Intel]

Hi BSung8,

Thanks for sharing again your initial results with us.

We’ll escalate this issue to Engineering team for further advice.

We shall update you once we’ve obtained feedback from them.

Regards,

Munesh

[Munesh_Intel]

Hi BSung8,

Thank you for your patience.

We require more inputs to investigate further.

Please provide the following additional information:

• The GPU driver version used for Ubuntu 20.04.
• The GPU driver version used for Yocto.

Regards,

Munesh

[BSung8]

Hi Munesh,

Sorry for late reply!

IOTG had updated TGL yocto source as MR4.  I have to build up New version Yocto image and re-test.

Please give me more time and feedback test result later.

[BSung8 · Accepted solution]

Hi Munesh,

The problem was fixed in version MR4.

As my environment :

TGL yocto:  release-81_tgl_u_mr4

OpenVINO : 2021.4.2

For your reference:

http://wt-intel.com/2021/12/12/%e6%96%87%e6%9b%84ai%e9%96%8b%e7%99%bc%e5%b7%a5%e5%85%b7/

Share with you.

Thanks!!!

[Munesh_Intel]

Hi BSung8,

Thank you for sharing with us.

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question.

Regards,

Munesh