The problem with export tensorflow model of fp16

[kang__alex]

Expected Behavior

      It is work on GPU or Movidius with FP16 Model

Current Behavior

    The result text display erro

Steps to Reproduce (for bugs)

     export the FP16 model

    get the bin and xml file

    put the file to the correct place.

Context

I can not get the right result .

My Environment

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):ubuntu 16.04

TensorFlow installed from (source or binary):binary

TensorFlow version (use command below):1.10.0

python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"

Python version:3.5

CUDA/cuDNN version:9.0 7

GPU model and memory:

The model can worked on CPU with export FP32,but can not worked on MYRiad

Is anybody could help me。

[Shubha_R_Intel]

Dear kang, alex,

Whether a model fully works or not on GPU or Myriad, while it works perfectly fine on CPU, fully depends on whether the layers of the model are also supported on GPU or Myriad. 

Please review the Supported Devices list to see if all your model's layers are supported (or not) for GPU and Myriad.

Hope it helps,

Thanks !

Shubha

[Hou_y_1]

Shubha R. (Intel) wrote:

Dear kang, alex,

Whether a model fully works or not on GPU or Myriad, while it works perfectly fine on CPU, fully depends on whether the layers of the model are also supported on GPU or Myriad. 

Please review the Supported Devices list to see if all your model's layers are supported (or not) for GPU and Myriad.

Hope it helps,

Thanks !

Shubha

Hi, Axpy Layer( caffe) is supported with fp16 on gpu?  I tested a caffemodel with Axpy layer, but a part of feature is null. I tested the same data with the other model without Axpy, I get right results. 

[kang__alex]

Thank you for your reply, i followed this link https://github.com/opencv/openvino_training_extensions/blob/develop/tensorflow_toolkit/lpr/README.md , training and export model,

but the fp32 model worked, and fp16 model cannot work. is there any special config for mo.py (eg. scale value, mean_value ) should be set for the export, I can not find any config in this link https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html

and when I use cross_check tools for those models(fp32 fp16)，it said “ unsupported primitive of type: ctcgreedydecoder name :ctcgreedydecoder” ，could you tell me what should I do？ 

Shubha R. (Intel) wrote:

Dear kang, alex,

Whether a model fully works or not on GPU or Myriad, while it works perfectly fine on CPU, fully depends on whether the layers of the model are also supported on GPU or Myriad. 

Please review the Supported Devices list to see if all your model's layers are supported (or not) for GPU and Myriad.

Hope it helps,

Thanks !

Shubha

[kang__alex]

Hi，I tested a tensorflow with fp16 on gpu？I think there is not the Axpy Layer in my model。

Hou y. wrote:

Quote:

Shubha R. (Intel) wrote:

Dear kang, alex,

Whether a model fully works or not on GPU or Myriad, while it works perfectly fine on CPU, fully depends on whether the layers of the model are also supported on GPU or Myriad. 

Please review the Supported Devices list to see if all your model's layers are supported (or not) for GPU and Myriad.

Hope it helps,

Thanks !

Shubha

Hi, Axpy Layer( caffe) is supported with fp16 on gpu?  I tested a caffemodel with Axpy layer, but a part of feature is null. I tested the same data with the other model without Axpy, I get right results. 

[Shubha_R_Intel]

Dear kang, alex,

you are correct. Axpy Layer is not supported. However you can add it by making a custom layer.  Here's a kinda old Custom Layer IDZ Post on how to do that but the content is still relevant. Redirect to the 2019 part of the repo for the references.

Also take a look at the below documentation:

https://docs.openvinotoolkit.org/latest/_docs_IE_DG_Integrate_your_kernels_into_IE.html

Hope it helps.

Thanks !

Shubha

[kang__alex]

Thank you for your reply, but there is not Axpy Layer in my modle, the error message said CTCGreedydecoder is not supported, but  I found this layer is supported on this link http://docs.openvinotoolkit.org/latest/_docs_IE_DG_supported_plugins_Supported_Devices.html

but when I use cross_check tools for those models(fp32 and fp16)，it said “ unsupported primitive of type: ctcgreedydecoder name :ctcgreedydecoder” ，could you tell me what should I do？ 

Thanks !

Alex

Shubha R. (Intel) wrote:

Dear kang, alex,

you are correct. Axpy Layer is not supported. However you can add it by making a custom layer.  Here's a kinda old Custom Layer IDZ Post on how to do that but the content is still relevant. Redirect to the 2019 part of the repo for the references.

Also take a look at the below documentation:

https://docs.openvinotoolkit.org/latest/_docs_IE_DG_Integrate_your_kernels_into_IE.html

Hope it helps.

Thanks !

Shubha

[Shubha_R_Intel]

Dear kang, alex, instead of cross_check_tool.py can you tell me what benchmark_app does ? There's a Python and C++ version. Please try it and report the results here. And you are correct,  CTCGreedydecoder is definitely supported.

Thanks,

Shubha

[kang__alex]

Dear Shubha R.

       The benchmar_app log  message is :

Progress: [....................] 100.00% done

[Step 8/8] Dump statistics report
[ INFO ] Statistics collecting was not requested. No reports are dumped.
Progress: [....................] 100.00% done

"

and the attachment is my model an xml file .could you help me to check it ?

Shubha R. (Intel) wrote:

Dear kang, alex, instead of cross_check_tool.py can you tell me what benchmark_app does ? There's a Python and C++ version. Please try it and report the results here. And you are correct,  CTCGreedydecoder is definitely supported.

Thanks,

Shubha