Running 2 Yolov4 Pipelines simultaneously causing wrong/no inference output on the second pipeline

[NikhilP]

Hi, 

Below are our platform details:

Platform : AWS m5dn.4xlarge
Intel OpenVINO : /opt/intel/openvino_2021.4.752/
RAM : 64GB
Intel CPU : Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz (w/ DL boost)
Inferencing device - CPU only

Issue:
- Trying 2 simultaneous pipelines one is doing Yolov4 Mask / No Mask detection and the other is doing Yolov4 Object detection. There is hardly any delay between the two pipelines when they get started (refer script,py.txt).

Observation:
- For the first pipeline (Yolov4 Mask / No-mask model), the output gets watermarked and bounding boxes are seen - [Output Video file attached].
- For the second pipeline (Yolov4 Object detection model), there is no inferencing output. Seems like the model is not loaded well and/or we are seeing corrupted inferencing - [Output Video file attached].
- Checked for logs to narrow down the issue but no errors thrown by the pipeline.

Attached:
- script.py.txt (pls rename it to .py file) - to reproduce the issue run - python3 script.py
- model_proc.zip - contains the json files as needed by the pipeline 
- Input video files
- Output inferenced video (showing the inferenced outputs)

We wanted to share the model files also but since they are 250 MB each (IR converted models) we are not able to attach the same. Kindly let us know how we can share it across for your reproduction. 

What we have also noticed is, if we run 1 pipeline at a time the inferencing works fine the issue only happens when both pipelines co-exist.

If you can get some clue on the problem by looking at the model proc files, do let us know if there are any corrections to be made else, we may have to share the model files with you for a reproduction at your end.

Regards,

Nikhil

[Wan_Intel]

Hi NikhilP,

Thanks for reaching out to us.

Please attached the required files to the email below for replication purposes:

waix.fook.wan@intel.com

Regards,

Wan

[NikhilP]

Hi Wan,

I have sent the large attachment to your email id. Kindly acknowledge if you have received it. 

Thank you,

Nikhil

[Wan_Intel]

Hi Nikhil,

Thanks for your information.

I've received the required files. We'll investigate this issue and we'll update you in the earliest.

Regards,

Wan

[NikhilP]

Thank you for the confirmation. One thing I forgot to mention is that we see this issue in Docker and bare metal machines that has DLStreamer installed. 

[Wan_Intel]

Hi NikhilP,

I encountered "Failed to construct OpenVINOImageInference" when running script.py in Ubuntu 20.04 with the latest version of Intel® Deep Learning Streamer (Intel® DL Streamer).

Could you please try your application with the latest version of Intel® Deep Learning Streamer (Intel® DL Streamer) and see if your application were able to run successfully?

Regards,

Wan

[NikhilP]

Hi Wan,

As mentioned earlier we are still on OpenVINO 2021.4.2 release version. All the models were created for the same base. 

Would it be possible for you to try -> openvino/ubuntu20_data_runtime:2021.4.2?


Regards,

Nikhil

[Wan_Intel]

Hi Nikhil,

Thanks for your information.

I encountered the same issue as you when running two YOLOv4 pipelines simultaneously.

One pipeline:

One pipeline:

Two Pipelines simultaneously:

We'll further investigate this issue and we'll update you at the earliest.

Regards,

Wan

[NikhilP]

Hi Wan,

Thank you, it's good that you are able to reproduce this. Will wait for your inputs.

Regards,

Nikhil

[NikhilP]

Hi Wan,

Do we have any findings or update for this problem ?

Regards,

Nikhil

[Wan_Intel]

Hi Nikhil,

Thanks for your patience.

We're still investigating this issue with our next level, and we'll update here as soon as possible.

Regards,

Wan​

[NikhilP]

Hi Wan,

Do you have any updates from your team ? 

Regards,

Nikhil

[Wan_Intel]

Hi Nikhil,

Thanks for your patience.

Our developers are still investigating this issue and we'll update you as soon as possible.

Regards,

Wan

[Wan_Intel]

Hi Nikhil,

Referring to the following thread:

https://community.intel.com/t5/Intel-Distribution-of-OpenVINO/gvawatermark-unable-to-adapt-the-meta-information-when-input/m-p/1474788

We noticed that you have upgraded the DL streamer on your local machine. Did you encounter issues when running two YOLOv4 pipelines simultaneously with the latest DL streamer?

Regards,

Wan

[NikhilP]

Hi Wan,

I was able to run 2-Yolov5 (simultaneous) pipelines in DLStreamer 2022.3 and it works.

I haven't tried 2-Yolov4 pipeline with DLStreamer 2022.3.

Regards,

Nikhil

[Wan_Intel]

Hi NikhilP,

We're glad to know that you were able to run 2-Yolov5 (simultaneous) pipelines in DLStreamer 2022.3. Meanwhile, do you still need help with the issue?

Regards,

Wan

[NikhilP]

Hi Wan,

We would need support on this thread to know why did 2 Yolov4 pipelines not work in 2021.4.2. We might not have immediate plans of upgrading to 2022.3 and hence would like to know how to debug and root cause this problem on 2021.4.2.

I was expecting we will hear some information from your next level team. Are we inferring that 2021.4.2 version cannot support Yolov4 models ?

Regards,

Nikhil 

[Wan_Intel]

Hi NikhilP,

Thanks for your question.

Please submit a new question if additional information is needed as this thread will no longer be monitored.

Regards,

Wan