Intel® Distribution of OpenVINO™ Toolkit
Community support and discussions about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all things computer vision-related on Intel® platforms.

Model compilation: NOT IMPLEMENTED

nwidynski
Novice
782 Views

Hi,

I'm trying to compile a model. I tried both compile_tool and the Export function, and they lead to the same result:

Inference Engine:
        API version ............ 2.1
        Build .................. 2021.2.0-1877-176bdf51370-releases/2021/2
        Description ....... API

Network inputs:
    Placeholder : FP32 / NCHW
    Placeholder_1 : FP32 / NCHW
    Placeholder_2 : FP32 / NCHW
Network outputs:
    generator/Conv/BiasAdd/Add : FP32 / NCHW

[NOT_IMPLEMENTED]

I only tried on Windows for now. Could I get some visibility on the not implemented layers? Is there a workaround (not compiling is not option for us because of the loading time on GPU)?

The network is in attachment.

Thanks in advance,

 

0 Kudos
1 Solution
nwidynski
Novice
638 Views

For anyone interested, we cannot use import/export at the moment for GPU.

However, the solution was to give Openvino a cache directory to store the compiled cl kernels. This way, the kernels are generated the first time your application runs, and then will be loaded from the same directory for additional runs. This should speed-up the loading.

ie.SetConfig( { { CONFIG_KEY( CACHE_DIR ), "your_path" } }, "GPU" );

 

View solution in original post

14 Replies
Iffa_Intel
Moderator
750 Views

Greetings,


The OpenVINO Compile tool is a C++ application that enables you to dump a loaded executable network blob. The tool is delivered as an executable file that can be run on both Linux* and Windows*.


you may refer here: https://docs.openvinotoolkit.org/latest/openvino_inference_engine_tools_compile_tool_README.html


and this is guide on how to Compile the Inference-Engine Samples :

https://www.youtube.com/watch?v=6Ww_zLDGfII


Sincerely,

Iffa



nwidynski
Novice
746 Views

Hi Iffa,

Thanks for your answer. Unfortunately that is not my question.

I know what the compile tool is, and in fact my message is the output of this compile tool. My problem is that the compile tool returns a "NOT IMPLEMENTED" and I can't figure out why and/or what is not implemented from my network.

I posted a network so you can reproduce the problem on your end.

Thanks,

Iffa_Intel
Moderator
743 Views

Hi,

I'm using OpenVINO version 2021.2 with Windows 10.

I'm able to use your file with the compile tool. However please take note that it's a myriad_compile.exe.

You can refer to my attachment.

nwidynski
Novice
740 Views

Hi Iffa,

Ok for Myriad. But it does not help in my case for CPU and GPU, does it? Using the compile tool, I'm getting "NOT_IMPLEMENTED" for both CPU and GPU. It outputs an empty blob file.

I'm also using OpenVINO version 2021.2 with Windows 10. I also tried with linux and it leads to the same result.

Thanks

Iffa_Intel
Moderator
736 Views

Can you show me the location of the compile tool that you are using, together with the complete command that you use?



Sincerely,

Iffa


nwidynski
Novice
733 Views
Iffa_Intel
Moderator
727 Views

We are investigating this further and will update you as soon as possible.



Sincerely,

Iffa


nano
Beginner
702 Views

Hey,

I ran into the same issue with an optimized onnx model in .bin/.xml form that can be compiled with the MYRIAD flag, but not with the cpu flag.

Just to say this once: I run an azure maschine with a xeon E5-2690, maybe there is some azure-sided virtualization in the background that messes with the -d "CPU" Flag?

Do you also use a cloud computer @nwidynski ? Did you happen to find a workaround yet?

Cheers

nwidynski
Novice
685 Views

Hi @nano ,

I haven't found a workaround/solution yet. I tried the compile tool on multiple local computers, -d CPU, -d GPU, and same results (no cloud computer).

My next step would be to compile OpenVino in Debug to generate the pdb and step into the code.

Cheers

nano
Beginner
671 Views

Hey @nwidynski ,

check out the github issue I opened for this problem: https://github.com/openvinotoolkit/openvino/issues/3863

Apparently compiling with the -d cpu flag is not supported currently. Compiling for CPU must be done during runtime using the IR.

 

Cheers

nwidynski
Novice
664 Views

Hi @nano ,

Thanks for the ticket and the link! Ok, it makes sense for CPU. Unfortunately, I'm mainly interested on the GPU case, since loading the network compiles it, and it takes more than 30 seconds to do so for that target. I don't see how a model can be deployed with that limitation.

Thanks,

nwidynski
Novice
639 Views

For anyone interested, we cannot use import/export at the moment for GPU.

However, the solution was to give Openvino a cache directory to store the compiled cl kernels. This way, the kernels are generated the first time your application runs, and then will be loaded from the same directory for additional runs. This should speed-up the loading.

ie.SetConfig( { { CONFIG_KEY( CACHE_DIR ), "your_path" } }, "GPU" );

 

View solution in original post

Iffa_Intel
Moderator
609 Views

Greetings,

 

Fyi, as you previously heard, the compile_tool only supports Myriad, FPGA for now and myriad_compile supports only Myriad.

 

documents will be updated accordingly by the developer soon.

 

Sincerely,

Iffa

 

Iffa_Intel
Moderator
595 Views

Greetings,


Intel will no longer monitor this thread since we have provided a solution. If you need any additional information from Intel, please submit a new question. 


Sincerely,

Iffa


Reply