DL workbench accuracy tool is failing by giving status as Accuracy Tool Failed for my faster rcnn inception v2 coco tensorflow object detection model v 1.15. Please help with the Configuration.
I have attached the screenshot of the DL workbench.
Thank you for the response.
when i'm trying to optimize model to INT8 using DL workbench, it's giving error as 'resize to input size possible, only for one input layer case'. i have attached screenshot also . Please help out in this issue.
Thank you for your question. If you need any additional information from Intel, please submit a new question as Intel is no longer monitoring this thread.
Hi @raushan ,
Starting from 2021.3 (released at the end of Q1 in 2021) DL Workbench has two modes of accuracy measurement: basic and advanced. https://docs.openvinotoolkit.org/latest/workbench_docs_Workbench_DG_Measure_Accuracy.html
- Basic mode provides simplified and reduced scope of accuracy settings and works for reduced range of models, for example basic mode supports models with only one input. At the same time
- Advanced mode allows to provide a valid accuracy config for the Accuracy Checker tool that is used inside. Therefore, the list of models that you can measure accuracy for is much larger and, for examples, allows to measure accuracy for models with several inputs.
In your particular case, in order to measure accuracy of Faster-RCNN model, you need to use Advanced mode and specify the input metadata layer in the configuration. Here is the example of configuration file that is valid for the following Faster-RCNN model (model with a name faster_rcnn_inception_v2_coco from https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf1_detection_zoo.md):
name: faster_rcnn_inception_v2_coco launchers: - adapter: type: ssd batch: 1 device: CPU framework: dlsdk inputs: - name: image_info type: CONST_INPUT value: - - 600 - 1024 - 1 model: $MODEL_PATH/faster_rcnn_inception_v2_coco.xml weights: $MODEL_PATH/faster_rcnn_inception_v2_coco.bin datasets: - annotation_conversion: annotation_file: $DATASET_PATH/annotations/instances_val2017_200pictures.json converter: mscoco_detection has_background: true images_dir: $DATASET_PATH/images use_full_label_map: true data_source: $DATASET_PATH/images metrics: - max_detections: 100 presenter: print_vector type: coco_precision name: dataset postprocessing: - dst_height: 600 dst_width: 1024 type: faster_rcnn_postprocessing_resize preprocessing: - aspect_ratio_scale: fit_to_window dst_height: 600 dst_width: 1024 type: resize - dst_height: 600 dst_width: 1024 pad_type: right_bottom type: padding subsample_size: 100%
Please note that you might want to specify additional parameters, refer to the Accuracy Checker documentation: https://github.com/openvinotoolkit/open_model_zoo/tree/master/tools/accuracy_checker
If you have any additional questions or proposals, please ask them via the official DL Workbench Discussion Forum: https://github.com/openvinotoolkit/workbench_feedback/discussions.