Yolov4 Model Stops Inference after few days of Operation

[NikhilP]

Hi, 

I have stumbled upon a strange issue where the Yolov4 Object Detection model (converted to IR) stops inferencing after few days of operation.

VA_SERVING_BASE version is : openvino/ubuntu20_data_runtime:2021.4.2

However when the issue happens, we reset the instance and things come back to normal. The next call we are able to see the inference happening.  Unfortunately I do not have logs enabled to check if the pipeline throws an error when we hot this situation after multiple days.

While looking at the release note of OpenVINO 2022.3, I see a similar issue reported:

JIRA Id: 86683
Description: App fails to inference after 1 month of operation
Component: OpenCL driver
Workaround: Try to update to the latest OpenCL driver 

Could you share me the link to understand the above issue ? I do want to know if the above issues are same and that even we should try a load the openCL driver in our docker file also. If so can you recommend the driver version for 2021.4.2.

Regards,

Nikhil

[Wan_Intel]

Hi NikhilP,

Thanks for reaching out to us.

Are you using GPU plugin when inferencing YOLOv4 Object Detection Model on Intel® Distribution of OpenVINO™ toolkit Docker image for Ubuntu 20.04 LTS? The recommended OpenCL driver version for Intel® Distribution of OpenVINO™ toolkit Docker image for Ubuntu 20.04 LTS is 19.41.14441.

On another note, please use OpenCL driver version 20.35.17767 for 10th generation Intel® Core™ processor (formerly Ice Lake) or 11th generation Intel® Core™ processor (formerly Tiger Lake). Please refer to Use a Docker Image for GPU in Install Intel® Distribution of OpenVINO™ toolkit for Linux from a Docker Image for more information

Regards,

Wan

[NikhilP]

Hi Wan,

Thanks for your quick response.

The Yolov4 pipeline is constructed to do inferencing on CPU's only for now. We have seen that the issue gets reproduced on Cascade Lake CPUs (Xeon 8250CL @ 2.5 GHz) and other Xeon processors. It's also clear that the docker file is not loading any OpenCL driver. 

Do you still recommend that we try loading the OpenCL drivers as a workaround to get over this issue? It also seems that the OpenCL driver version is CPU specific. In case the direction is to go ahead and load a specific OpenCL driver version which one would that be. 

Regards,

Nikhil

[Wan_Intel]

Hi NikhilP,

Thanks for your information.

Referring to the information provided, the YOLOv4 pipeline was constructed to do inferencing on CPU only, it does not load the OpenCL driver. Please use the latest version of Intel® Distribution of OpenVINO™ toolkit Docker image for Ubuntu 20.04 LTS and see if the issue can be resolved.

Regards,

Wan

[Wan_Intel]

Hi NikhilP,

Thanks for your question.

If you need additional information from Intel, please submit a new question as this thread will no longer be monitored.

Regards,

Wan