Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6418 Discussions

hang at FreeLibrary even after ov::shutdown()

anony
Beginner
2,629 Views

This is a continuation of the question below.

using FreeLibrary() and (language: C++ version: 2021.4.2 LTS),ERROR 

I call my own .dll from call.exe and use opnenvino from this own .dll.

 

As I asked about openvino before, I changed the version to 2022, ov::shutdown() and tried to unload the own.dll with FreeLibrary, but it hangs at FreeLibrary.

When I searched for minimal elements as the cause of this, the following appeared to be the cause.

ov::CompiledModel compiled_model;

compiled_model = ie.compile_model(network, device_name);

ov::InferRequest infer_request = compiled_model.create_infer_request();

 

In this code only inferRequest is defined as global.

After doing 「infer_request = compiled_model.create_infer_request();」, ov::shtudown(), even unloading with FreeLibrary hangs. However, if I didn't call 「infer_request = compiled_model.create_infer_request(); 」

it wouldn't hang (of course, I can't infer anything either).

 

where is the mistake in my code? Please let me know if there is any information I need to reproduce.

Or if there is a way to free or destroy the inferrequest and compilation model.

0 Kudos
1 Solution
Hairul_Intel
Moderator
1,955 Views

Hi anony,

Thank you for your patience.

 

As for now, we do not have a sample code that meets your requirement, the advice that our developer could provide is to suggest using shared pointers to help hold and control the lifetime for global objects.

 

Basically, the initial concept that our developer provided is to destroy the global InferRequest object before calling ov::shutdown.

 

We apologize for any inconvenience this has caused.

 

 

Regards,

Hairul


View solution in original post

0 Kudos
18 Replies
Hairul_Intel
Moderator
2,604 Views

Hi anony,

Thank you for reaching out to us.

 

Could you please provide the code that you're using so that we can replicate this issue? If it's not possible, you can also share a simplified version of it so that we can investigate this from our end.

 

 

Regards,

Hairul


0 Kudos
anony
Beginner
2,598 Views

thank you for your reply.

Unfortunately I can't give you the code.
Instead, I would like to ask, has Intel confirmed that the DLL can be freed using FreeLibrary after ov::shutdown?
In my case (already written some sample code above) it hangs.

Please refer to sample.cpp on the following site for an outline of the code. (You can see it by clicking sample.cpp in the site)
Sorry for Japanese.

 

https://qiita.com/**bleep**/items/38f1bb8fb4fc42d4fd3b

 

Can FreeLibrary free a DLL containing such code?

If it has been confirmed, please let me know the code and configuration in that case.
I'm testing with an object detection model and a cnn simple classification model.

0 Kudos
Hairul_Intel
Moderator
2,579 Views

Hi anony,

We're investigating this matter and will update you on any findings as soon as possible.

 

On a side note, which specific OpenVINO version are you using from the 2022 release (e.g. 2022.1.0, 2022.1.1, 2022.2 or 2022.3)?

 

 

Regards,

Hairul


0 Kudos
anony
Beginner
2,576 Views

thank you

 

The version I'm using is w_openvino_toolkit_windows_2022.3.1.9227.cf2c7da5689_x86_64.

0 Kudos
Hairul_Intel
Moderator
2,528 Views

Hi anony,

Thank you for your patience.

 

From our end, we have validated and tested the functionality of ov::shutdown(). Here is the test result for running ov::shutdown() in hello_classification sample:

 ovshutdown.png

 

If we exclude ov::shutdown() there'll be a module which remains loaded (openvino_ir_frontend.dll). Here we can see that there's no unload process for the openvino_ir_frontend.dll if we do not call the ov::shutdown() function:

 noshutdown.png

 

The possible reason for your issue with FreeLibrary is that there may be some application being scheduled periodically which is instantiating the access to to the DLL.

 

Unfortunately, we can only provide guidance regarding this issue since we do not have access to the source code.

 

 

Regards,

Hairul

 

0 Kudos
anony
Beginner
2,516 Views

Thank you for answering. I prepared the code based on hello_classification.cpp, so please check it.

This code hangs at the FreeLibrary part.

 

//source.cpp/////////////////////////////////////////////////////

#include "pch.h"
#include "source.h"
#include <iostream>
#include <vector>
#include <windows.h>
#include <opencv2/opencv.hpp>
#include <openvino/openvino.hpp>

ov::InferRequest infer_request;
int WINAPI dll() {
try {
// -------- Step 1. Initialize OpenVINO Runtime Core --------
ov::Core core;

std::string model_path = model_path;
cv::Mat img = cv::imread(img_path);


std::shared_ptr<ov::Model> model = core.read_model(model_path);

ov::element::Type input_type = ov::element::u8;
ov::Shape input_shape = { (unsigned long)1, (unsigned long)img.rows, (unsigned long)img.cols, (unsigned long)3 };

// just wrap image data by ov::Tensor without allocating of new memory
ov::Tensor input_tensor = ov::Tensor(input_type, input_shape, img.data);

const ov::Layout tensor_layout{ "NHWC" };

// -------- Step 4. Configure preprocessing --------

ov::preprocess::PrePostProcessor ppp(model);

ppp.input().tensor().set_shape(input_shape).set_element_type(input_type).set_layout(tensor_layout);

ppp.input().preprocess().resize(ov::preprocess::ResizeAlgorithm::RESIZE_LINEAR);
ppp.input().model().set_layout("NHWC");
ppp.output().tensor().set_element_type(ov::element::f32);
model = ppp.build();

ov::CompiledModel compiled_model = core.compile_model(model, "CPU");

infer_request = compiled_model.create_infer_request();
infer_request.set_input_tensor(input_tensor);
infer_request.infer();
const ov::Tensor& output_tensor = infer_request.get_output_tensor();

// -----------------------------------------------------------------------------------------------------

ov::shutdown();
}
catch (const std::exception& ex) {
std::cerr << ex.what() << std::endl;
return EXIT_FAILURE;
}

 

return EXIT_SUCCESS;
}

///////////////////////////////////////////////////////

#pragma once

#include <windows.h>

#ifdef EXPORTS
#define API __declspec(dllexport)
#else
#define API __declspec(dllimport)
#endif


extern "C" API int dll();


///////////////////////////////////////////////////////
DLL_call.cpp

#include <iostream>
#include <windows.h>
#include "source.h"

typedef int(*FUNC_dll1)();
int main()
{
HMODULE hModule = LoadLibraryA("Dll1.dll");

FUNC_dll1 called_dll1 = (FUNC_dll1)GetProcAddress(hModule, "dll");

called_dll1();
int error = FreeLibrary(hModule);

std::cout << error << std::endl;

return 0;
}

0 Kudos
Hairul_Intel
Moderator
2,458 Views

Hi anony,

Thank you for sharing the code.

 

We've escalated this issue to the relevant teams and will update you on any findings as soon as possible.

 

 

Regards,

Hairul


0 Kudos
anony
Beginner
2,363 Views

I would be grateful if you could let us have your answer 

0 Kudos
Hairul_Intel
Moderator
2,346 Views

Hi anony,

We just got feedback from the Developer team. Before calling the ov::shutdown you need to destroy all OpenVINO objects.

 

For example:

// Some function which works with OpenVINO

void openvino_inference() {

ov::Core core;

ov::Tensor tensor;

}

// After the return from the openvino_inference() function all openvino objects

// and shared ptrs will be destroyed

 

int main() {

openvino_inference();

ov::shutdown();

return 0;

}

 

// Also you can use the next approach

int main() {

{

ov::Core core;

ov::Tensor tensor;

}

ov::shutdown();

return 0;

}

 

 

Regards,

Hairul


0 Kudos
anony
Beginner
2,255 Views

Thank you for your reply.

How can I explicitly destroy an openvino object?
In your answer you were destroying the object by going outside the scope of the variable.
In my project's case I need to secure ov::InferRequest infer_request globally, so I need another method than making it unscoped.

Otherwise, I think that it will not be possible to perform inference using a DLL that incorporates openvino at any time.

0 Kudos
Hairul_Intel
Moderator
2,236 Views

Hi anony,

We've conveyed your concerns to the relevant teams for further investigation and will update you on any findings as soon as possible.

 

 

Regards,

Hairul


0 Kudos
anony
Beginner
2,135 Views

We look forward to hearing back from you. 

 

Best regards

0 Kudos
Hairul_Intel
Moderator
2,127 Views

Hi anony,

Thank you for your patience.

 

We just got feedback from our Developer team regarding this issue.

 

Our Developer suggests using share pointers which can help to hold and control life time for global objects.

 

Basically, if you have a global InferRequest or CompileModel and wants to call ov::shutdown, you should destroy these objects before the call. One possible way is to use shared pointers and reset them before calling ov::shutdown to destroy all OV objects.

 

 

Regards,

Hairul


0 Kudos
anony
Beginner
1,993 Views

thank you.

 

Please give me an example of code to control life time for global objects other than going out of scope.

 

Best regards

0 Kudos
Hairul_Intel
Moderator
1,986 Views

Hi anony,

We're checking with the relevant teams regarding this issue and will get back to you as soon as possible.



Regards,

Hairul


0 Kudos
Hairul_Intel
Moderator
1,956 Views

Hi anony,

Thank you for your patience.

 

As for now, we do not have a sample code that meets your requirement, the advice that our developer could provide is to suggest using shared pointers to help hold and control the lifetime for global objects.

 

Basically, the initial concept that our developer provided is to destroy the global InferRequest object before calling ov::shutdown.

 

We apologize for any inconvenience this has caused.

 

 

Regards,

Hairul


0 Kudos
anony
Beginner
1,886 Views

thank you
It was very helpful.

I hope intel will release the code, or I hope I will be able to release it myself.
I apologize for taking your time because I didn't know about spared_ptr.

Thank you.

0 Kudos
Hairul_Intel
Moderator
1,811 Views

Hi anony,

This thread will no longer be monitored since we have provided information. If you need any additional information from Intel, please submit a new question.

 

 

Regards,

Hairul


0 Kudos
Reply