- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
My objective is to optimize and calibrate my model for INT8 usage.
After saving the model from tensorflow2 to the saved_model format,
the input dimension is (None, 128,128,1). At the import step at the workbench,
I set the dimensions to (1,128,128,1) for batch 1 and I set the image colorspace
to grayscale. Processing with 11 files in the standard mode works fine
an gives measurement results.
My test files for "Segmantic Segmentation" are in the "Common Semantic Segmentation" format.
Can I specify for int8-Calibration that the jpg-Files are imported as grayscale?
The jpg-files in css-format are grayscale (only 1 component)
"file pic1.jpg: JPEG image data, JFIF standard 1.01, aspect ratio, density 1x1, segment length 16, baseline, precision 8, 640x480, components 1"
After that I tried to optimize with tpot (online and a local linux). There I got the following error message
The -result-archive argument is not set. The default value /home/workbench/.workbench/models/2/21/job_artifacts/artifact.tar.gz will be used.
[setupvars.sh] OpenVINO environment initialized
0%| |00:00
0%| |00:00
4%|▍ |00:00Traceback (most recent call last):
File "/usr/local/bin/pot", line 11, in <module>
load_entry_point('pot==1.0', 'console_scripts', 'pot')()
File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/app/run.py", line 36, in main
app(sys.argv[1:])
File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/app/run.py", line 60, in app
metrics = optimize(config)
File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/app/run.py", line 138, in optimize
compressed_model = pipeline.run(model)
File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/compression/pipeline/pipeline.py", line 54, in run
result = self.collect_statistics_and_run(model, current_algo_seq)
File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/compression/pipeline/pipeline.py", line 64, in collect_statistics_and_run
model = algo.run(model)
File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/compression/algorithms/quantization/default/algorithm.py", line 94, in run
self.algorithms[1].algo_collector.compute_statistics(model)
File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/compression/statistics/collector.py", line 70, in compute_statistics
_, stats_ = self._engine.predict(combined_stats, sampler)
File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/compression/engines/ac_engine.py", line 166, in predict
stdout_redirect(self._model_evaluator.process_dataset_async, **args)
File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/compression/utils/logger.py", line 129, in stdout_redirect
res = fn(*args, **kwargs)
File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/libs/open_model_zoo/tools/accuracy_checker/accuracy_checker/evaluators/quantization_model_evaluator.py", line 142, in process_dataset_async
self._fill_free_irs(free_irs, queued_irs, infer_requests_pool, dataset_iterator, **kwargs)
File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/libs/open_model_zoo/tools/accuracy_checker/accuracy_checker/evaluators/quantization_model_evaluator.py", line 329, in _fill_free_irs
batch_input, batch_meta = self._get_batch_input(batch_inputs, batch_annotation)
File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/libs/open_model_zoo/tools/accuracy_checker/accuracy_checker/evaluators/quantization_model_evaluator.py", line 80, in _get_batch_input
filled_inputs = self.input_feeder.fill_inputs(batch_input)
File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/libs/open_model_zoo/tools/accuracy_checker/accuracy_checker/launcher/input_feeder.py", line 193, in fill_inputs
inputs = self.fill_non_constant_inputs(data_representation_batch)
File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/libs/open_model_zoo/tools/accuracy_checker/accuracy_checker/launcher/input_feeder.py", line 187, in fill_non_constant_inputs
filled_inputs, extract_image_representations(data_representation_batch, meta_only=True)
File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/libs/open_model_zoo/tools/accuracy_checker/accuracy_checker/launcher/input_feeder.py", line 324, in _transform_batch
self.precision_mapping.get(layer_name)
File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/libs/open_model_zoo/tools/accuracy_checker/accuracy_checker/launcher/dlsdk_launcher.py", line 840, in fit_to_input
return self._align_data_shape(data, layer_name, layout)
File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/libs/open_model_zoo/tools/accuracy_checker/accuracy_checker/launcher/dlsdk_launcher.py", line 606, in _align_data_shape
return data.reshape(input_shape) if not self.disable_resize_to_input else data
ValueError: cannot reshape array of size 49152 into shape (1,1,128,128)
=> As 49152/128/128 = 3, it seems that the dimension is somehow set to 3? RGB?
Any hints, which settings I've to check?
Thank you
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Andreas Fehlner,
Thank you for reaching out to us.
Please share with us your model and the dataset so that we can investigate this issue further.
Sincerely,
Zul
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
I've attached the requested data.
Thank you
Andreas
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
I tried setting up a correct configuration file for the accuracy checker.
(I also tried out
a)
reader:
type: opencv_imread
reading_flag: gray
b)
#preprocessing:
- type: bgr_to_gray
=> I think this should not be correct as the original image only contains one channel
My best guess is attached:
The error log is:
andi@er:~/workT/currproj$ accuracy_check -c accuracy_css_gray2.yml
['models']
Processing info:
model: moba
launcher: dlsdk
device: CPU
dataset: d500
OpenCV version: 4.5.3-openvino
Annotation conversion for d500 dataset has been started
Parameters to be used for conversion:
converter: common_semantic_segmentation
images_dir: /home/andi/workT/currproj/css/Images
masks_dir: /home/andi/workT/currproj/css/Masks
image_postfix: .png
mask_postfix: .png
dataset_meta_file: /home/andi/workT/currproj/css/dataset_meta.json
Annotation conversion for d500 dataset has been finished
IE version: 2021.4.0-3839-cd81789d294-releases/2021/4
Loaded CPU plugin version:
CPU - MKLDNNPlugin: 2.1.2021.4.0-3839-cd81789d294-releases/2021/4
Found model /home/andi/workT/currproj/saved_model.xml
Found weights /home/andi/workT/currproj/saved_model.bin
Input info:
Layer name: input_1
precision: FP32
shape [1, 1, 128, 128]
Output info
Layer name: StatefulPartitionedCall/model/conv2d_25/BiasAdd/Softmax
precision: FP32
shape: [1, 3, 128, 128]
0%| | 0/495 [00:00<?]23:14:46 accuracy_checker ERROR: cannot reshape array of size 49152 into shape (1,1,128,128)
Traceback (most recent call last):
File "/opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/main.py", line 419, in main
evaluator.process_dataset(
File "/opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/evaluators/model_evaluator.py", line 334, in process_dataset_sync
filled_inputs, batch_meta = self._get_batch_input(batch_annotation, batch_input)
File "/opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/evaluators/model_evaluator.py", line 235, in _get_batch_input
filled_inputs = self.input_feeder.fill_inputs(batch_input)
File "/opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/launcher/input_feeder.py", line 193, in fill_inputs
inputs = self.fill_non_constant_inputs(data_representation_batch)
File "/opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/launcher/input_feeder.py", line 186, in fill_non_constant_inputs
return self._transform_batch(
File "/opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/launcher/input_feeder.py", line 321, in _transform_batch
batch_data[layer_name] = self.input_transform_func(
File "/opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/launcher/dlsdk_launcher.py", line 840, in fit_to_input
return self._align_data_shape(data, layer_name, layout)
File "/opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/launcher/dlsdk_launcher.py", line 606, in _align_data_shape
return data.reshape(input_shape) if not self.disable_resize_to_input else data
ValueError: cannot reshape array of size 49152 into shape (1,1,128,128)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Andreas Fehlner,
Thank you for sharing the model and the dataset with us. Can we know where is the source of your model and which TensorFlow 2 model that you are using (eg: EfficientDet, ssd_mobilenet, etc)?
Sincerely,
Zul
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Zul,
thank you for the answer!
The original model (a unet/sdunet) was custom made in tensorflow 2.4.1/2.4.2 and saved as 'SavedModel'.
The variant attached here is a very reduced and not trained version, which shows the same behavoir as the original model.
The input images are grayscale.
Sincerely,
Andreas
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Andreas Fehlner,
We can optimize the model to INT8 and run the accuracy test. The current workaround is you need to change the preprocessing to bgr_to_gray, in order for it to work without error.
Sincerely,
Zul
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Zul,
thank you for investigating the issue. This really helped a lot.
I also realized that the missing postprocessing part in my yml-file produced unexpected behavoir.
At the moment different issues exist (running workbench with openvino 2021.4):
1) No accuracy values although an annotated file is used (Common semantic segmentation)
2) Although I've done int8 optimization, all slices remained FP32.
Thank you
Andreas
My steps were the following:
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Andreas Fehlner,
Thank you for sharing your steps. To run the accuracy checker and convert to INT8, please follow these steps:
Run the Accuracy Check:
- Click on Configure Accuracy
- Change usage to semantic segmentation
- Run Accuracy Check
If it results in “ValueError: cannot reshape array of size 49152 into shape (1,1,128,128)”
Reconfigure the accuracy checker setting by:
- Click on configure accuracy
- Change from Basic to Advanced mood
- modify the preprocessing:
- type: bgr_to_gray
- Run Accuracy Check
Optimize to INT8 precision
- Click on Perform
- INT8
- Optimize
- Select the annotated dataset
- Optimize
Run the Accuracy Check:
- Click on Configure Accuracy
- Change from Basic to Advanced mood
- modify the preprocessing:
- type: bgr_to_gray
- Run Accuracy Check
Sincerely,
Zulkifli
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Andreas Fehlner,
This thread will no longer be monitored since we have provided a solution. If you need any additional information from Intel, please submit a new question.
Sincerely,
Zulkifli

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page