Compile the saved model of Google Cloud AutoML Vison with Compile Tool

[taka]

Hello,

When compiling the attached IR file with Compile Tool, the following error occurs.
The attached IR file can be executed on the CPU or GPU.

Is there a way to compile this IR model with Compile Tool?
(If it is not currently supported, I hope that Compile Tool will be improved in the future.)

[Execution environment]
ubuntu 20.04.3LTS
openvino2021.4
model: Google Cloud AutoML Vision saved_model

---------------------------------------error message---------------------------------------------------

user@193f58464f81:~/workdir/$ ${INTEL_OPENVINO_DIR}/deployment_tools/tools/compile_tool/compile_tool \
> -m saved_model.xml \
> -ip U8 \
> -VPU_NUMBER_OF_SHAVES 4 \
> -VPU_NUMBER_OF_CMX_SLICES 4 \
> -o saved_model.blob \
> -d MYRIAD
Inference Engine:
IE version ......... 2021.4.0
Build ........... 2021.4.0-3839-cd81789d294-releases/2021/4

Network inputs:
mnas_v4_a_1/feature_network/stem/conv/Conv2D_Fold/placeholder_port_0 : U8 / NCHW
Network outputs:
Sigmoid : FP16 / NC
[Warning][VPU][Config] Deprecated option was used : VPU_MYRIAD_PLATFORM

[ GENERAL_ERROR ]
/home/jenkins/agent/workspace/private-ci/ie/build-linux-ubuntu20/b/repos/openvino/inference-engine/src/vpu/graph_transformer/src/frontend/frontend.cpp:439 Failed to compile layer "mnas_v4_a_1/feature_network/stem/conv/add_fold": [ GENERAL_ERROR ]
/home/jenkins/agent/workspace/private-ci/ie/build-linux-ubuntu20/b/repos/openvino/inference-engine/src/vpu/graph_transformer/src/stages/convolution.cpp:44 invalid number of inputs: 3u

------------------------------------------------------------------------------------------------------

[Peh_Intel]

Hi taka,

Thanks for reaching out to us and sharing your model with us.

I noticed that your model has been quantized as there are FakeQuantize layers in the model.

For your information, the low-precision 8-bit inference is optimized for Intel CPU and GPU only. You can refer to Low-Precision 8-bit Integer Inference for more detailed supported devices.

Hence, try to compile the FP16 IR model.

Regards,

Peh

[taka]

hi, Peh

The attached file is FP16 IR model.　
Just because there are FakeQuantize layers doesn't mean it's an 8-bit IR model.

The details of the error are as follows.

The problem is occurring in the "mnas_v4_a_1 / feature_network / stem / conv / add_fold" layer.

>  /home/jenkins/agent/workspace/private-ci/ie/build-linux-ubuntu20/b/repos/openvino/inference

> -engine/src/vpu/graph_transformer/src/frontend/frontend.cpp:439

> Failed to compile layer "mnas_v4_a_1/feature_network/stem/conv/add_fold": [ GENERAL_ERROR ]

> /home/jenkins/agent/workspace/private-ci/ie/build-linux-ubuntu20/b/repos/openvino/inference

> -engine/src/vpu/graph_transformer/src/stages/convolution.cpp:44 invalid number of inputs: 3u


Regards,

[Peh_Intel]

Hi taka,

The FakeQuantize layers are one of the unsupported layer types for the compilation. The "mnas_v4_a_1 / feature_network /stem/conv/add_fold" layer is added before the FakeQuantize layers and hence the compilation failed due to the input size is not equal to 1.

For a separate note, I was able to compile an FP16 age-gender-recognition-retail-0013 model (without FakeQuantize layers). However, when I tried to compile an FP16-INT8 age-gender-recognition-retail-0013 model (with FakeQuantize layers), I obtained the similar error as you.

Regards,

Peh

[Peh_Intel]

Hi taka,

This thread will no longer be monitored since we have provided answer. If you need any additional information from Intel, please submit a new question. 

Regards,

Peh