Hey !
This post is linked with this Link Stackoverflow .
I would like to know what are exactly dependancies of MaskRCNN Demo ?
- I used OpenCV DNN library in order to create an efficient inference engine "CustomIE".
- I used successfully OpenVINO Model optimiser (python), to build the .xml and .bin file representing my network.
- I successfully builded OpenVINO Sample directory with Visual Studio 2017 and run MaskRCNNDemo project with my network.
Finally, i wanted to change my OpenCV DNN "CustomIE" by an openvino based inference engine...
I reproduced almost exactly the same code than in Mask RCNN Demo, but i have different results....
With MaskRCNN_Demo :
mask_rcnn_demo.exe -m .\Release\frozen_inference_graph.xml -i .\Release\input.jpg
InferenceEngine:
API version ............ 1.4
Build .................. 19154
[ INFO ] Parsing input parameters
[ INFO ] Files were added: 1
[ INFO ] .\Release\input.jpg
[ INFO ] Loading plugin
API version ............ 1.5
Build .................. win_20181005
Description ....... MKLDNNPlugin
[ INFO ] Loading network files
[ INFO ] Preparing input blobs
[ WARNING ] Image is resized from (4288, 2848) to (800, 800)
[ INFO ] Batch size is 1
[ INFO ] Preparing output blobs
[ INFO ] Loading model to the plugin
[ INFO ] Start inference (1 iterations)
Average running time of one iteration: 2593.81 ms
[ INFO ] Processing output blobs
[ INFO ] Detected class 16 with probability 0.986519: [2043.3, 1104.9], [2412.87, 1436.52]
[ INFO ] Image out.png created!
[ INFO ] Execution successful
Then everything work fine !
With my VinoBased IE :
Image is resized from (4288, 2848) to (800, 800) Detected class 62 with probability 1: [4288, 0], [4288, 0]
I tried to watch "deeper" in order to understand what can be the issue....
I first had a look on dependancies :
<MaskRCNNDemo>
//References
<format_reader/> => Open CV Images, resize it and get uchar data
<ie_cpu_extension/> => CPU extension for un-managed layers (?)
//Linker
format_reader.lib => Format Reader Lib (VINO Samples Compiled)
cpu_extension.lib => CPU extension Lib (VINO Samples Compiled)
inference_engined.lib => Inference Engine lib (VINO)
opencv_world401d.lib => OpenCV Lib
libiomp5md.lib => Dependancy
... (other libs)
On my own project :
<CustomIA> //References None => I use my own libtiff way to open image and i resize with OpenCV None => I will just add include to cpu_extension source code. //Linker opencv_world345d.lib => OpenCV 3.4.5 library tiffd.lib => Libtiff Library cpu_extension.lib => CPU extension compiled with sample inference_engined.lib => Inference engine lib.+ Following dll
cpu_extension.dll inference_engined.dll libiomp5md.dll mkl_tiny_omp.dll MKLDNNPlugind.dll opencv_world345d.dll tiffd.dll tiffxxd.dll
The cpu_extension.dll is the one compiled in the sample dir of OpenVino (idem for cpu_extension.lib.
Inference_engine.lib is the one from openvino directory.
I successfully compile but i face issue with plugin dispatcher :
Demo CODE:
slog::info << "Loading plugin" << slog::endl;
InferencePlugin plugin = PluginDispatcher({ FLAGS_pp, "../../../lib/intel64" , "" }).getPluginByDevice(FLAGS_d);
/** Loading default extensions **/
if (FLAGS_d.find("CPU") != std::string::npos) {
/**
* cpu_extensions library is compiled from "extension" folder containing
* custom MKLDNNPlugin layer implementations. These layers are not supported
* by mkldnn, but they can be useful for inferring custom topologies.
**/
plugin.AddExtension(std::make_shared<Extensions::Cpu::CpuExtensions>());
}
/** Printing plugin version **/
printPluginVersion(plugin, std::cout);
OUTPUT :
[ INFO ] Loading plugin
API version ............ 1.5
Build .................. win_20181005
Description ....... MKLDNNPlugin
My CODE:
VINOEngine::VINOEngine()
{
// Loading Plugin
std::cout << std::endl;
std::cout << "[INFO] - Loading VINO Plugin..." << std::endl;
this->plugin= PluginDispatcher({ "", "../../../lib/intel64" , "" }).getPluginByDevice("CPU");
this->plugin.AddExtension(std::make_shared<Extensions::Cpu::CpuExtensions>());
printPluginVersion(this->plugin, std::cout);
OUTPUT :
[INFO] - Loading VINO Plugin... 000001A242280A18 // Like memory adress ???
Second issue is when i try to get boxes :
When i try to extract my ROI and masks from New Code, if i have a "match", i always have :
score =1.0
x1=x2=0.0
y1=y2=1.0
But the mask looks well extracted...
MYCODE :
float score = box_info[2];
if (score > this->Conf_Threshold)
{
// On reconstruit les coordonnées de la box..
float x1 = std::min(std::max(0.0f, box_info[3] * Image.cols), static_cast<float>(Image.cols));
float y1 = std::min(std::max(0.0f, box_info[4] * Image.rows), static_cast<float>(Image.rows));
float x2 = std::min(std::max(0.0f, box_info[5] * Image.cols), static_cast<float>(Image.cols));
float y2 = std::min(std::max(0.0f, box_info[6] * Image.rows), static_cast<float>(Image.rows));
int box_width = std::min(static_cast<int>(std::max(0.0f, x2 - x1)), Image.cols);
int box_height = std::min(static_cast<int>(std::max(0.0f, y2 - y1)), Image.rows);
Do anybody have an idea about what i make badly ?
How to create and link correctly an OpenVINO project using cpu_extension ?
Thanks !
Link Copied
Shubha R. (Intel) wrote:Francois, may I ask how you are running your executable ? I assume it's a command line Windows app? Because in the command-line you do have to pass in a specific switch to add the full path cpu_extension.dll in order to actually execute the application. I see that you have statically linked cpu_extesion.lib with your application. Instead of doing that, can you try to load it through the command-line the way OpenVino samples do ? I think once you get your app working in this manner, you will be able to figure out why static linking doesn't work. There are probably some software pathways you are missing if you're not passing it in the DLL through the command-line. Then you can use something like Process Explorer to see if the DLL is actually being loaded.
-l "<absolute_path>" Optional. Absolute path to library with MKL-DNN (CPU) custom layers (*.so).
Hello, thanks for your answer !
I run my program directly from the degugger like that :
CustomIE.exe "PathtoImage.jpg"
The program contains a "main.cpp" like that :
int main(int argc,char** argv)
{
std::string FilePath;
VINOEngine MaskVino =VINOEngine();
//CNNEngine MaskCV = CNNEngine();
FilePath = argv[1];
bool isDisplayed = true;
if (argc > 2)
{
if (strcmp(argv[2],"-NoDisplay")==0)
{
isDisplayed = false;
}
}
//MaskCV.ProcessImage(FilePath,1);
MaskVino.ProcessImage("input.jpg", true,false);
return 0;
}
And i build the vino engine with the constructor like i described in the post just before.
How to pass the path of cpu_extension.dll ? Because i already call a "plugindispatcher" with path set to "./" containing a compiled cpu_extension.dll.
The problem is that i don't have any exception, it seems to work like a charm except that detection is bad and i don't seem to load plugin correctly unfortunately.
Unfortunately, in my app, i don't manage the -l args, because it is not based exactly on the "example".
I watched the code of mask_rcnn_sample, and if "-l" args is set, it seems to wait a ".so" file. But as you noticed, i'am on windows, then i can't manage such files this way.
I'll try to inspect with process Explorer.
Hope you'll find a way to help me :).
if it can help, i ll show you the way i linked my program :
Include
Linker
