OpenVINO: Issue with releasing NCS2 resources programmatically

[SaifVazir]

Hi, I'm trying to use the Neural Compute Stick 2 to perform inference using a CPP application on Windows 10. The application gets the model file and parameters dynamically and performs inference (let's stick to SSD models for now). I'm sort of using the async object detector sample code provided in OpenVINO samples. The code runs perfectly with the NCS2 when it gets initialized with default values, but it breaks with  "Can not init Myriad device: NC_ERROR" when a new user defined configuration is loaded in the same process. Upon further digging, I notice that the NCS2 or "MYRIAD" is not present when running GetAvailableDevices() after the new configuration arrives, probably because the resources were not freed up correctly. A simple way to reproduce this error is trying to run the object_detection_demo.exe in quick succession. (A probable reason could be that the NCS2 device path and id change when the model is loaded). I'm attaching the screenshots of running the object_detector_demo.exe and how the device instance path changes. 

I've tried setting the "MYRIAD_ENABLE_FORCE_RESET" to "YES" both on the Inference Engine Core object and during LoadNetwork, but it doesn't seem to work.

Any help is appreciated.

[Syamimi_Intel]

Hi Saif Vazir,

Thank you for reaching out. We are currently investigating this and will update you with the information soon. 

I need one information about your environment, which version of OpenVINO you used ?

Regards,

Syamimi

[SaifVazir]

Hi, I'm using OpenVINO version 2021.2.185.

[Syamimi_Intel]

Hi Saif Vazir,

Thank you for waiting. I have validate the object_detection_demo.exe with quick succession and I get same error as you:

[ ERROR ] Can not init Myriad device: NC_ERROR

As such, I also try with several experiments and come out with a different result when running other sample C++ code which have allocate() function implemented and there is no issue when with NCS when a quick succession is done.

For your information, to make the NCS to work perfectly for quick succession, a memory handling mechanism is required in C++ such as allocate() or ie_infer_request_free().

If you are running object_detection_demo.exe, you need to wait at least 10 seconds for another run.

Therefore, you can try to sample C++ code such as pedestrian_tracker_demo and security_barrier_camera for a quick succession run.

Regards,

Syamimi

[SaifVazir]

Hi, thanks for the reply. I'm actually using the AsyncPipeline and the ModelSSD classes included in async_pipeline.h and detection_model_ssd.h in my solution. These do not use any form of allocate()/infer request free in their model/pipeline execution or while setting input_blobs, do you think it's a better idea to classify this as a bug?

Also, ie_infer_request_free() is for the C API and not the C++ API so that wouldn't work for me. The pedestrian_tracker_demo does not use the async pipeline and isn't an asynchronous solution which is what I'm looking for.

I experimented with the segmentation_demo_async which also uses async_pipeline and it also throws an error when run in quick succession, this makes me believe that the pipeline class implementation is holding/not freeing up resources even after being destroyed. I'd be grateful if you can take a look into this and help me out. Thanks!

[Vladimir_Dudnik]

@SaifVazir we were able to reproduce the issue with OpenVINO 2021.3 Open Model Zoo C++ object_detection_demo, so confirm the issue exist for demos which utilize AsyncPipeline class. We will investigate and fix that on Open Model Zoo github and of course in the next OpenVINO release.

Thanks for noticing this issue.

Regards,
  Vladimir

[SaifVazir]

Thanks a lot @Vladimir_Dudnik. Is there any issue link on github that I can follow for updates on this issue? I've been working to identify it as well, so I might be able to provide a bit of help too.

[Vladimir_Dudnik]

@SaifVazir I think there is no associated issue on github yet. Would you mind to create one? That might be helpful for other, who meet this issue too.

[Vladimir_Dudnik]

@SaifVazir we've identified root cause of issue and are working on solution. Meantime, you may take a look at PR-2375 on Open Model Zoo github to see where it goes.

[SaifVazir]

Thanks for the quick reply and solution. I'll be sure to follow the PR for further updates and test it out myself to confirm that it resolves the issue. Thanks a lot again @Vladimir_Dudnik!

[Syamimi_Intel]

Hi Saif Vazir,

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question.

Regards,

Syamimi