Intel® Distribution of OpenVINO™ Toolkit
Community support and discussions about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all things computer vision-related on Intel® platforms.

Model compilation: NOT IMPLEMENTED

nwidynski
Novice
1,295 Views

Hi,

I'm trying to compile a model. I tried both compile_tool and the Export function, and they lead to the same result:

Inference Engine:
        API version ............ 2.1
        Build .................. 2021.2.0-1877-176bdf51370-releases/2021/2
        Description ....... API

Network inputs:
    Placeholder : FP32 / NCHW
    Placeholder_1 : FP32 / NCHW
    Placeholder_2 : FP32 / NCHW
Network outputs:
    generator/Conv/BiasAdd/Add : FP32 / NCHW

[NOT_IMPLEMENTED]

I only tried on Windows for now. Could I get some visibility on the not implemented layers? Is there a workaround (not compiling is not option for us because of the loading time on GPU)?

The network is in attachment.

Thanks in advance,

 

0 Kudos
1 Solution
nwidynski
Novice
1,151 Views

For anyone interested, we cannot use import/export at the moment for GPU.

However, the solution was to give Openvino a cache directory to store the compiled cl kernels. This way, the kernels are generated the first time your application runs, and then will be loaded from the same directory for additional runs. This should speed-up the loading.

ie.SetConfig( { { CONFIG_KEY( CACHE_DIR ), "your_path" } }, "GPU" );

 

View solution in original post

14 Replies
Iffa_Intel
Moderator
1,263 Views

Greetings,


The OpenVINO Compile tool is a C++ application that enables you to dump a loaded executable network blob. The tool is delivered as an executable file that can be run on both Linux* and Windows*.


you may refer here: https://docs.openvinotoolkit.org/latest/openvino_inference_engine_tools_compile_tool_README.html


and this is guide on how to Compile the Inference-Engine Samples :

https://www.youtube.com/watch?v=6Ww_zLDGfII


Sincerely,

Iffa



nwidynski
Novice
1,259 Views

Hi Iffa,

Thanks for your answer. Unfortunately that is not my question.

I know what the compile tool is, and in fact my message is the output of this compile tool. My problem is that the compile tool returns a "NOT IMPLEMENTED" and I can't figure out why and/or what is not implemented from my network.

I posted a network so you can reproduce the problem on your end.

Thanks,

Iffa_Intel
Moderator
1,256 Views

Hi,

I'm using OpenVINO version 2021.2 with Windows 10.

I'm able to use your file with the compile tool. However please take note that it's a myriad_compile.exe.

You can refer to my attachment.

nwidynski
Novice
1,253 Views

Hi Iffa,

Ok for Myriad. But it does not help in my case for CPU and GPU, does it? Using the compile tool, I'm getting "NOT_IMPLEMENTED" for both CPU and GPU. It outputs an empty blob file.

I'm also using OpenVINO version 2021.2 with Windows 10. I also tried with linux and it leads to the same result.

Thanks

Iffa_Intel
Moderator
1,249 Views

Can you show me the location of the compile tool that you are using, together with the complete command that you use?



Sincerely,

Iffa


Iffa_Intel
Moderator
1,240 Views

We are investigating this further and will update you as soon as possible.



Sincerely,

Iffa


nano
Beginner
1,215 Views

Hey,

I ran into the same issue with an optimized onnx model in .bin/.xml form that can be compiled with the MYRIAD flag, but not with the cpu flag.

Just to say this once: I run an azure maschine with a xeon E5-2690, maybe there is some azure-sided virtualization in the background that messes with the -d "CPU" Flag?

Do you also use a cloud computer @nwidynski ? Did you happen to find a workaround yet?

Cheers

nwidynski
Novice
1,198 Views

Hi @nano ,

I haven't found a workaround/solution yet. I tried the compile tool on multiple local computers, -d CPU, -d GPU, and same results (no cloud computer).

My next step would be to compile OpenVino in Debug to generate the pdb and step into the code.

Cheers

nano
Beginner
1,184 Views

Hey @nwidynski ,

check out the github issue I opened for this problem: https://github.com/openvinotoolkit/openvino/issues/3863

Apparently compiling with the -d cpu flag is not supported currently. Compiling for CPU must be done during runtime using the IR.

 

Cheers

nwidynski
Novice
1,177 Views

Hi @nano ,

Thanks for the ticket and the link! Ok, it makes sense for CPU. Unfortunately, I'm mainly interested on the GPU case, since loading the network compiles it, and it takes more than 30 seconds to do so for that target. I don't see how a model can be deployed with that limitation.

Thanks,

nwidynski
Novice
1,152 Views

For anyone interested, we cannot use import/export at the moment for GPU.

However, the solution was to give Openvino a cache directory to store the compiled cl kernels. This way, the kernels are generated the first time your application runs, and then will be loaded from the same directory for additional runs. This should speed-up the loading.

ie.SetConfig( { { CONFIG_KEY( CACHE_DIR ), "your_path" } }, "GPU" );

 

Iffa_Intel
Moderator
1,122 Views

Greetings,

 

Fyi, as you previously heard, the compile_tool only supports Myriad, FPGA for now and myriad_compile supports only Myriad.

 

documents will be updated accordingly by the developer soon.

 

Sincerely,

Iffa

 

Iffa_Intel
Moderator
1,108 Views

Greetings,


Intel will no longer monitor this thread since we have provided a solution. If you need any additional information from Intel, please submit a new question. 


Sincerely,

Iffa


Reply