INT8 quantization of ssd mobilenet v1 model

[CJOY01]

I am trying to convert ssd mobilenet v1 FP32 model to INT8 using calibration tool. I am using openvino version 2019_R3.1 and tensorflow version  1.1.4.0,   

I am facing one issue on using the   calibrate.py tool used for  calibrating the FP32 model.

for calibrating the model, i tried the below mentioned step,

python3 calibrate.py --config /opt/intel/openvino_2019.3.376/deployment_tools/tools/calibration_tool/configs/ssd_mobilenet_v1_coco.yml --definition /opt/intel/openvino_2019.3.376/deployment_tools/tools/calibration_tool/configs/definitions.yml -M /opt/intel/openvino_2019.3.376/deployment_tools/model_optimizer --tf_custom_op_config_dir ~/tf_custom_op_configs --models ~/models/ssd  -a ~/annotations --converted_models ~/models/ssd/ -e /home/cyril.pj/Downloads/OPENVINOSTANDALONEREFERNECE/Openvino_3ddfa_2019_R3.1/bin/intel64/Release/lib/ --source /home/cyril.pj/Downloads/IMAGENET_DTAASET_155GB/imagenet-object-localization-challenge/imagenet_object_localization_patched2019/ILSVRC/Data/CLS-LOC/val

but i am not able to successfully execute the calibration tool, error i am getting is ,

=================================

[ ERROR ]  Failed to match nodes from custom replacement description with id 'ObjectDetectionAPIPreprocessorReplacement':
It means model and custom replacement description are incompatible.
Try to correct custom replacement description according to documentation with respect to model node names
[ ERROR ]  Failed to match nodes from custom replacement description with id 'ObjectDetectionAPISSDPostprocessorReplacement':
It means model and custom replacement description are incompatible.
Try to correct custom replacement description according to documentation with respect to model node names

[ SUCCESS ] Generated IR model.
[ SUCCESS ] XML file: /home/cyril.pj/models/ssd/ssd_mobilenet_v1_coco.xml
[ SUCCESS ] BIN file: /home/cyril.pj/models/ssd/ssd_mobilenet_v1_coco.bin
[ SUCCESS ] Total execution time: 8.57 seconds. 
[ INFO ] CPU extensions is loaded /home/cyril.pj/Downloads/OPENVINOSTANDALONEREFERNECE/Openvino_3ddfa_2019_R3.1/bin/intel64/Release/lib/libcpu_extension.so
17:54:39 openvino.tools.calibration INFO: Processor: x86_64
17:54:39 openvino.tools.calibration INFO: Collecting original network statistics for /home/cyril.pj/models/ssd/ssd_mobilenet_v1_coco.xml...
Traceback (most recent call last):
  File "calibrate.py", line 21, in <module>
    network = calibration.Calibrator(config).run()
  File "/opt/intel/openvino_2019.3.376/python/python3.5/openvino/tools/calibration/calibrator.py", line 251, in run
    raw_results = self.collect_fp32_results()
  File "/opt/intel/openvino_2019.3.376/python/python3.5/openvino/tools/calibration/calibrator.py", line 53, in collect_fp32_results
    collect_performance_counters=True)
  File "/opt/intel/openvino_2019.3.376/python/python3.5/openvino/tools/calibration/base_calibrator.py", line 343, in infer
    model_evaluator = ModelEvaluator.from_configs(launcher_config, dataset_config)
  File "/opt/intel/openvino_2019.3.376/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/evaluators/model_evaluator.py", line 57, in from_configs
    dataset = Dataset(dataset_config)
  File "/opt/intel/openvino_2019.3.376/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/dataset.py", line 66, in __init__
    raise ConfigError('path to converted annotation or data for conversion should be specified')
accuracy_checker.config.config_validator.ConfigError: path to converted annotation or data for conversion should be specified
========================================

According to my analysis, issue is with annotation file, annotation file required for ssd mobilenet V1 is not found in the specified annotation folder.

please see the below section of  ssd_mobilenet_v1_coco.yml  file,

   datasets:
      # uniquely distinguishable name for dataset
      # note that all other steps are specific for this dataset only
      # if you need to test topology on multiple datasets, you need to specify
      # every step explicitly for each dataset
      - name: COCO2017_90cl_bkgr

        # list of metrics, calculated on dataset
        metrics:
          - type: map
            integral: 11point
            ignore_difficult: True
            presenter: print_scalar

          - type: coco_precision
 

here , the dataset  mobilenetv1 is trying to get is  different, i checked the openvino tools page, and only abe to found the INT8 quantization steps for inception model only, there we were using Imagenet dataset,  kindly provide the necessary steps for making the annotation files required for ssd_mobilenet_v1 model  and steps to convert the ssd_mobilenet_v1 FP32 model to INT8 model using calibration tool.

[Max_L_Intel]

Hello Cyril.

Please kindly check the similar thread with steps recommended for ssd_mobilenet_v1_coco INT8 quantization using previous build of OpenVINO toolkit - https://software.intel.com/en-us/forums/intel-distribution-of-openvino-toolkit/topic/810212

And also please have a chance to learn and test a completely new calibration tool within Post-Training Optimization toolkit as a part of OpenVINO toolkit latest 2020.1 build - http://docs.openvinotoolkit.org/latest/_README.html
You can download it here https://software.intel.com/en-us/openvino-toolkit/choose-download