Unable to get correct classification results after converting from tensorflow classification model

[NikhilParmar]

Hi Team,

[Scenario]

I have used lobe.ai to create a classification model, took the export for TensorFlow out and converted to openvino format using the mo_tf.

TensorFlow Directory Structure

 

Command used for conversion to openvino format

 python3 /opt/intel/openvino_2021.4.582/deployment_tools/model_optimizer/mo_tf.py --saved_model_dir {INPUT_DIR}/ --output_dir {OUTPUT_DIR} --data_type FP32 --batch 1 --reverse_input_channels --scale 255

MO Logs Attached

signature.json Attached

classification script 

Output from script

[ INFO ] Creating Inference Engine
[ INFO ] Reading the network: theatre_output/saved_model.xml
[ INFO ] Configuring input and output blobs
[ INFO ] Number of classes 2
[ INFO ] Loading the model to the plugin
[ WARNING ] Image theatre_classification/2.jpeg is resized from (128, 67) to (224, 224)
[ INFO ] Starting inference in synchronous mode
[ INFO ] Image path: theatre_classification/2.jpeg
[ INFO ] Top 10 results:
[ INFO ] classid probability
[ INFO ] -------------------
[ INFO ] 1       1.0000000
[ INFO ] 0       0.0000000
[ INFO ]
[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool

Issue:

For every image I try using normal python script and loading TensorFlow models I get the desired result, but with every image the output is exactly the same. I am unable to determine the issue here.

[Iffa_Intel]

Hi,

Could you share/attach your model files here? (native,IR,etc) if possible

Sincerely,

Iffa

[NikhilParmar]

Here is the uploaded link.

Contents:

1. Base Tensorflow related model files

2. Openvino Exported Files

[NikhilParmar]

Hi @Iffa_Intel Any update on this?

[Iffa_Intel]

We are currently investigating this and will get back to you asap

Sincerely,

Iffa

[Iffa_Intel]

I had tested your model and it seems okay since I didn't see any problem popping out. (You may refer to my attachments.)

I can see the hello_classification does give out some output as you did. Probably you need to further train the model according to what you are trying to achieve, hence producing more variants.

Another way is, you could try out the pre-trained OpenVINO models: 

https://docs.openvinotoolkit.org/latest/omz_models_group_intel.html

Sincerely,

Iffa

[NikhilParmar]

Hi @Iffa_Intel ,

Thanks for the update!

A basic 101 question when I tested the original model, It was working fine infact this has been the case with 3 different classification models I tested.

[NikhilParmar]

@Iffa_Intel I had tested again with 2 different models and there is definitely a drop in accuracy for example:

I trained a fruits classification model with 6-7 classes. With the original results being 98% accurate a picture of banana is being called out as guava for the openvino classification and banana by the original model. The image which I downloaded from the internet.

[Iffa_Intel]

From Model Optimizer (conversion to IR) perspective, I believe there is no issue.

Since you custom build this model, I believe something is not being properly implemented or some algorithm is insufficient/missing in the native model that you wrote the code.

The best way is for you to try and study the validated model & apply it to yours.

You may refer to officially supported models & topology here: https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html

Sincerely,

Iffa

[Iffa_Intel]

Greetings,

Intel will no longer monitor this thread since we have provided a solution. If you need any additional information from Intel, please submit a new question.

Sincerely,

Iffa