- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
So I'd successfully convert my model into INT8 format, after this, I'm trying to use the accuracy checker to see whether my model will perform.
I ran the following command line
$ python3 accuracy_check.py \
--config /opt/intel/openvino_2019.3.376/deployment_tools/open_model_zoo/tools/accuracy_checker/configs/resnet-50.yml \
--models /opt/intel/openvino_2019.3.376/deployment_tools/open_model_zoo/tools/downloader/public/resnet-50/ \
--source /home/ubuntu/imagenet/val/ \
-a /home/ubuntu/imagenet/annotations/imagenet/ \
-td CPU \
-e /opt/intel/openvino_2019.3.376/deployment_tools/inference_engine/lib/intel64/libcpu_extension_avx2.so \
-M /opt/intel/openvino_2019.3.376/deployment_tools/model_optimizer/ \
--definitions /opt/intel/openvino_2019.3.376/deployment_tools/tools/calibration_tool/configs/definitions.yml
however the result is
Result:
15:25:29 accuracy_checker WARNING: /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a sy
nonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
15:25:29 accuracy_checker WARNING: /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a sy
nonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
15:25:29 accuracy_checker WARNING: /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a sy
nonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
15:25:29 accuracy_checker WARNING: /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a sy
nonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
15:25:29 accuracy_checker WARNING: /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a sy
nonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
15:25:29 accuracy_checker WARNING: /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a sy
nonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
15:25:29 accuracy_checker WARNING: /usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type'
as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
15:25:29 accuracy_checker WARNING: /usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type'
as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
15:25:29 accuracy_checker WARNING: /usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type'
as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
15:25:29 accuracy_checker WARNING: /usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type'
as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
15:25:29 accuracy_checker WARNING: /usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type'
as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
15:25:29 accuracy_checker WARNING: /usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type'
as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
Processing info:
model: resnet-50-cf
launcher: caffe
device: CPU
dataset: imagenet_1000_classes
OpenCV version: 4.1.2-openvino
Traceback (most recent call last):
File "accuracy_check.py", line 19, in <module>
main()
File "/opt/intel/openvino_2019.3.376/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/main.py", line 202, in main
model_evaluation_mode(config, progress_reporter, args)
File "/opt/intel/openvino_2019.3.376/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/main.py", line 218, in model_evaluation_mode
model_evaluator = ModelEvaluator.from_configs(launcher_config, dataset_config)
File "/opt/intel/openvino_2019.3.376/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/evaluators/model_evaluator.py", line 57, in from_configs
dataset = Dataset(dataset_config)
File "/opt/intel/openvino_2019.3.376/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/dataset.py", line 66, in __init__
raise ConfigError('path to converted annotation or data for conversion should be specified')
accuracy_checker.config.config_validator.ConfigError: path to converted annotation or data for conversion should be specified
-----------------------------------------------------------------------------
Clearly I'm missing something, but i can't figure out what is missing.
I also provide the file context below to make it easier to know my situation.
resnet-50.yml:
models:
- name: resnet-50-cf
launchers:
- framework: caffe
model: public/resnet-50/resnet-50.prototxt
weights: public/resnet-50/resnet-50.caffemodel
adapter: classification
datasets:
- name: imagenet_1000_classes
preprocessing:
- type: resize
size: 256
aspect_ratio_scale: greater
- type: crop
size: 224
- type: normalization
mean: 104, 117, 123
- name: resnet-50
launchers:
- framework: dlsdk
tags:
- FP32
model: public/resnet-50/FP32/resnet-50.xml
weights: public/resnet-50/FP32/resnet-50.bin
adapter: classification
- framework: dlsdk
tags:
- FP16
model: public/resnet-50/FP16/resnet-50.xml
weights: public/resnet-50/FP16/resnet-50.bin
adapter: classification
datasets:
- name: imagenet_1000_classes
preprocessing:
- type: resize
size: 256
aspect_ratio_scale: greater
- type: crop
size: 224
The definitions.yml file is from https://software.intel.com/en-us/forums/computer-vision/topic/807243
Thanks
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi whung6,
Thank you for contacting Intel Customer Support.
I am currently working on your inquiry, I will reply you back as soon as possible.
If you have any other questions, feel free to ask.
Regards,
David C.
Intel Customer Support Technician
A Contingent Worker at Intel
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi whung6,
I apologize for the delay in our response, looking further into the error message. It looks like you may need to update the definitions.yml to ensure data_source is pointing to the path to the directory with input data and the annotation is pointing to the pickle file.
Let us know if you need further assistance.
Best Regards,
David C.
Intel Customer Support Technician
A Contingent Worker at Intel
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for the solution, I did mispoint to the wrong directory, after fixing the problem I can now work well.
![](/skins/images/D3C0B914909A6564BBB97F4AD1ED1973/responsive_peak/images/icon_anonymous_message.png)
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page