Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Stevenys_L_
Beginner
175 Views

[OpenVINO 2018.R4] After converted inception_v3, openvino result is different with tensorflow

Hi,

I follow this issue(https://software.intel.com/en-us/forums/computer-vision/topic/799700) 
and Intel tutorial(https://software.intel.com/en-us/articles/OpenVINO-Using-TensorFlow) 
to convert my own model(inception_v3) to .xml & .bin,

but the output result top_1~top_5 is different with the original tensorflow
(reference: "result/classify_flower_openvino_r4.csv", "result/classify_flower_tf.csv")

Here is my steps (Train the flower datasets from tensorflow-slim):

- Tensorflow slim steps:

1. Using TensorFlow-Slim to fine-tuning a model from an existing checkpoint

   export DATASET_DIR=/<MY_PATH>/slim_flower/flowers
   export TRAIN_DIR=/<MY_PATH>/slim_flower/inception_v3
   export CHECKPOINT_PATH=/<MY_PATH>/slim_flower/inception_v3.ckpt

   python train_image_classifier.py \
        --train_dir=${TRAIN_DIR} \
        --dataset_dir=${DATASET_DIR} \
        --dataset_name=flowers \
        --dataset_split_name=train \
        --model_name=inception_v3 \
        --checkpoint_path=${CHECKPOINT_PATH} \
        --checkpoint_exclude_scopes=InceptionV3/Logits,InceptionV3/AuxLogits \
        --trainable_scopes=InceptionV3/Logits,InceptionV3/AuxLogits

2. Then it will generate .ckpt and .pbtxt (in openvino_issue/inception_v3_flower):
       - model.ckpt-1535333.data-00000-of-00001 
       - model.ckpt-1535333.index
       - model.ckpt-1535333.meta
       - graph.pbtxt

3. Dwonload tensorflow_models, and put the annex code in slim project

$ git clone https://github.com/tensorflow/models.git tensorflow_models
$ cp -r openvino_issue/flower_photos /<PATH>/tensorflow_models/research/slim/
$ cp -r openvino_issue/inception_v3_flower /<PATH>/tensorflow_models/research/slim/
$ cp openvino_issue/classify_flower_tf_slim.py /<PATH>/tensorflow_models/research/slim/

4. Start inference

$ python3 classify_flower_tf_slim.py \
      --checkpoint_path=openvino_issue/inception_v3_flower/model.ckpt-1535333 \
      --test_list=./flower_list.txt \
      --test_dir=openvino_issue/flower_photos \
      --batch_size=1 \
      --num_classes=5 \
      --model_name=inception_v3

- OpenVINO R4 steps:

1. Cause of .pbtxt has no input placeholder, so I use openvino_issue/model_freeze_graph.py to generate it

  $ cd /<PATH>/tensorflow_models/research/slim/
  $ cp openvino_issue/model_freeze_graph.py /<PATH>/tensorflow_models/research/slim/
  $ python3 model_freeze_graph.py

    Then it will generate (in openvino_issue/openvino_step_1 folder) :
       - flower_inception_v3.ckpt.data-00000-of-00001
       - flower_inception_v3.ckpt.index
       - flower_inception_v3.ckpt.meta
       - new_graph.pbtxt

2. freeze model

  $ cd /opt/intel/computer_vision_sdk_2018.4.420/deployment_tools/model_optimizer
  $ mkdir -p openvino_issue/inception_v3_flower

  ## copy flower_inception_v3.ckpt.data-00000-of-00001, flower_inception_v3.ckpt.index,  flower_inception_v3.ckpt.meta, new_graph.pbtxt into openvino_issue/inception_v3_flower ##

  $ python3 /usr/local/lib/python3.5/dist-packages/tensorflow/python/tools/freeze_graph.py \
      --input_graph openvino_issue/inception_v3_flower/new_graph.pbtxt \
      --input_checkpoint openvino_issue/inception_v3_flower/flower_inception_v3.ckpt \
      --output_node_names InceptionV3/Predictions/Reshape_1 \
      --output_graph flower_frozen_inception_v3.pb

3. check topology

  $ python3 /opt/intel/computer_vision_sdk_2018.4.420/deployment_tools/model_optimizer/mo/utils/summarize_graph.py --input_model ./flower_frozen_inception_v3.pb
       
  1 input(s) detected:
  Name: input, type: float32, shape: (-1,299,299,3)
  1 output(s) detected:
  InceptionV3/Predictions/Reshape_1

4. convert .xml & .bin

$ python3 /opt/intel/computer_vision_sdk_2018.4.420/deployment_tools/model_optimizer/mo_tf.py \
       --input_model /opt/intel/computer_vision_sdk_2018.4.420/deployment_tools/model_optimizer/flower_frozen_inception_v3.pb \
       --input_shape [1,299,299,3] \
       --mean_value [127.5,127.5,127.5] \
       --scale 127.5 \
       --input input \
       --model_name flower_inception_v3_r4 \
       --output_dir /volume1/home/admin/stevenys/classify_flower

5. The classifiation of openvino code : classify_flower_openvino_r4.py
     command: $ python3 classify_flower_openvino_r4.py (in openvino_issue.zip)

    
<p.s.> If csv of result cannot open by excel, you can execute code again
       1. $ python3 test_image_classifier.py ......
       2. $ python3 classify_flower_openvino_r4.py

 

The results are not the same, do you have any suggest ?
Or in OpenVINO step 1 have something wrong?

Thank you for your help.

Here is my file link(276MB):
https://drive.google.com/file/d/1VOUb9mJGZj7dyQzuDgcHaJVHADjpFpGk/view?usp=sharing

 

0 Kudos
4 Replies
kumar__sumit
Beginner
175 Views

I am trying to do inference on open vino after retraining the inceptionV3 model. I was following the steps you mentioned above but in OpenVINO R4 steps 1, I couldn't find openvino_issue/model_freeze_graph.py file. Could you please tell me where can i find this file or the way to add input placeholder layer in the .pbtxt file generated earlier.

Thanks,

sumit

Shubha_R_Intel
Employee
175 Views

freeze_graph.py is a Tensorflow product and you can find it here (see below link). If you installed Tensorflow on your computer though, there should be a local version on your machine.

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph.py

smith__jet
Beginner
175 Views

hello, i tried to calculate fp32 accuracy using Openvino R5 "validation_app" but it produced this:
[ INFO ] Average infer time (ms): 64.45 (15.52 images per second with batch size = 1)                   │classification_sample_async  InferenceEngineUnitTests         speech_sample
Top1 accuracy: 1.02% (51 of 5000 images were detected correctly, top class is correct)                  
Top5 accuracy: 16.94% (847 of 5000 images were detected correctly, top five classes contain required 

Well , i think the results should be like this:
top1 accuracy: 80.2%(it is tested using tensorflow).

Here is my run condition:
**OS**: Ubuntu 16.04
**project version**: Openvino R5(2019 branch)
**pretrained model**: **inception_v4 tensorflow model** [https://github.com/tensorflow/models/tree/master/research/slim](url)
**validation dataset** : imagenet 2012(i use one tenth of total images and 5 images each class)

Here are my experimental steps:

**1. Converting a TensorFlow Model**
under the dldt/model-optimizer, run mo_tf.py, the following is my sh script:

> python3 mo_tf.py \                                                                                                                                                                                    
>          --input_model  ./inception_v4.pb \                                                                                                                   
>          --model_name Inception_v4 \                                                                                                                                                                     
>          --output_dir inception_v4 \                                                                                                                                                                     
>          --input_shape [1,299,299,3] \                                                                                                                                                                   
>          --data_type FP32 \                                                                                                                                                                              
>          --mean_value [127.5,127.5,127.5] \                                                                                                                                                              
>          --scale 127.5 \                                                                                                                                                                                 
>          --reverse_input_channels \                                                                                                                                                                      
>          --input input_img \                                                                                                                                                                             
>          --output InceptionV4/Logits/Logits/BiasAdd

output:
Model Optimizer arguments:
Common parameters:
        - Path to the Input Model:      ****inception_v4.pb
        - Path for generated IR:        ****/inception_v4
        - IR output name:       Inception_v4
        - Log level:    ERROR
        - Batch:        Not specified, inherited from the model
        - Input layers:         input_img
        - Output layers:        InceptionV4/Logits/Logits/BiasAdd
        - Input shapes:         [1,299,299,3]
        - Mean values:  [127.5,127.5,127.5]
        - Scale values:         Not specified
        - Scale factor:         127.5
        - Precision of IR:      FP32
        - Enable fusing:        True
        - Enable grouped convolutions fusing:   True
        - Move mean values to preprocess section:       False
        - Reverse input channels:       True
TensorFlow specific parameters:
        - Input model in text protobuf format:  False
        - Path to model dump for TensorBoard:   None
        - List of shared libraries with TensorFlow custom layers implementation:        None
        - Update the configuration file with input/output node names:   None
        - Use configuration file used to generate the model with Object Detection API:  None
        - Operations to offload:        None
        - Patterns to offload:  None
        - Use the config file:  None
Model Optimizer version:        unknown versio

it generate three files:
Inception_v4.bin  Inception_v4.mapping  Inception_v4.xml

**2. run validation_app** 
first build the project under dldt/inference-engine/, then validation_app binaries could be found in inference-engine/bin/intel64/Release, run the shell like this:
```
./validation_app -t C \                                                                                                                                                                                 
          -i ***imagenet_val/5000_image/img_class.txt \                                                                                                                         
          -m ****/dldt/model-optimizer/inception_v4/Inception_v4.xml \                                                                                                            
          -d CPU \                                                                                                                                                                                        
          -ppType ResizeCrop \                                                                                                                                                                            
          -ppSize 299     
```                                                                                                                                                                               
  
my image list formats as: 
  ILSVRC2012_val_00005901.JPEG 97                                                                                                                                                                        
  ILSVRC2012_val_00006460.JPEG 97                                                                                                                                                                        
  ILSVRC2012_val_00001602.JPEG 406                                                                                                                                                                       
  ILSVRC2012_val_00002523.JPEG 406                                                                                                                                                                       
  ILSVRC2012_val_00002876.JPEG 406   

well, 5000 images are placed under imagenet_val/5000_image. 
i have checked all my steps and confused about why the top1 acc is only **1.02%**!!!!!!

Shubha_R_Intel
Employee
175 Views

Dear smith, jet and kumar, sumit,

First if you haven't already done so, please upgrade to 2019 R1 ! Next, reading your steps above, I don't think you used model optimizer properly to create IR for an Inception model. We have very specific steps to do that in the below link:

https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_...

Thanks for using OpenVino !

Shubha

 

Reply