How to do async or sync openvino object detection and evaluation with GUI (Without loop)?

[emregun]

1.Project Summary:

I want to optimise my Tensorflow trained faster_rcnn_inception_v2_coco model object detection project with openvino toolkit. My model works totally fine with following samples: object_detection_demo_ssd_async and object_detection_sample_ssd.

In my project, I have a folder with photos and  do object detection (2 fps) on these photos which can be started and stopped via GUI.

2.Problem:

I could not achieve to integrate openvino into a system with GUI (without loop since  GUI is incompatible with loop.)

3.What I tried before:

3.1Thread:
3.1.1.Pseudo Code:
-Configure GUI
-Configure Object detection and openvino  
-Start_thread_with_openvino_object_detection_and_evaluation   (which includes a function with a loop for openvino object detection and evaluation(similar like object_detection_demo_ssd_async, but everything is done in a loop))
-Start_GUI 
3.1.2.Thread Problem:
Code works fine, it is possible to start and stop in gui, but after a couple of   minutes, object detection getting slower (Slow down From 1 fps to 15 fps). I had also the same problem with Tensorflow(without Openvino) I guess the problem is that threads are not capable to work in parallel in python, so after a while the processor cannot manage the GUI and object detection together, so it  slows down. 

3.2. Multiprocessing solution 1(This solution works fine with Tensorflow without openvino):
3.2.1.Pseudo Code:
The same pseudo code as thread, but instead of thread, function with object detection and evaluation loop is called from multiprocessing
3.2.2.Multiprocessing solution 1 Problem:  
After following method in the loop, 
exec_net.start_async(request_id=next_request_id, inputs=data)
it is not possible to call following method(or any method which comes from exec_net object)
exec_net.requests[cur_request_id].wait(-1) == 0
The code freezes, if any method from exec_net ist called after exec_net.start_async(Although the loop works totally fine if it is not called via multiprocessing). My assumption is that the detection cannot be finished in the process, so therefore the exec_net is busy all the time. Therefore I tried the second multiprocessing solution below calling object detection and evaluation in separate processes.

3.3.Multiprocessing solution 2: (Separate multiprocess call for object detection and evaluation)
3.3.1.Pseudo Code:
-Configure GUI
-Configure Object detection and openvino  
-Start_timer1(which calls every sec a new multiprocessing for a single object detection(exec_net.start_async) 
-start_timer2(which calls every sec a new multiprocessing for a single detection evaluation(for obj in output_postprocessor(exec_net.requests[cur_request_id].output_blobs))
-Start GUI
3.3.2.Multiprocessing solution 2 Problem:  
It is not possible to share so complex objects(exec_net) between processes (from object detection process to evaluation process) in python. I also tried to use evaluation methods directly in main function(Instead of start_timer2) but I could also not achieve to share complex objects(exec_net) to main function from the first object detection process.

4.Environment:
OS:macOS Catalina
Processor:1,6 GHz Dual-Core Intel Core i5
Programming Language:Python
Model: Tensorflow trained faster_rcnn_inception_v2_coco
Samples: object_detection_demo_ssd_async and object_detection_sample_ssd 

I hope the description above is not too complex. If you need the exact code examples instead of Pseudo Code, I am ready to share them.

Thank you very much in advance.

Best Regards

[emregun]

Correction:

3.1.2.Thread Problem:
Slow down From 1 fps to 15 fps->Slow down almost 15x times

[Iffa_Intel]

Greetings,

Before proceeding further, I need to confirm which GUI are you using for development? Is it GUI for C++ like Qt?

or probably webservice GUI like I did here (it's for Python but the concept should be the same): https://github.com/SNNJM/People_Counter

Sincerely,

Iffa

[Iffa_Intel]

Greetings,

Before proceeding further, I need to confirm which GUI are you using for development? Is it GUI for C++ like Qt?

or probably webservice GUI like I did here (it's for Python but the concept should be the same): https://github.com/SNNJM/People_Counter

Sincerely,

Iffa

[emregun]

Hi Iffa_Intel,

Thanks a lot for your response. I have a python based PyQt5 GUI, which will be running on macOS.

Best Regards

Emre

[Iffa_Intel]

In that case, you might want to refer here: https://github.com/itlab-vision/openvino-smart-library

and see how you can integrate OpenVINO and PyQT

This is our official documentation on how to Integrate the Inference Engine with Your Application

It's general but perhaps it could help.

Sincerely,

Iffa

[emregun]

Hi,

Thank you very much for your response. I will try and let you know about  the results.

Have nice week.

Best regards
Emre

[Iffa_Intel]

Noted.

Sincerely,

Iffa

[Iffa_Intel]

Greetings,

Intel will no longer monitor this thread since we have provided a solution. If you need any additional information from Intel, please submit a new question. 

Sincerely,

Iffa