Int8 optimization failed (wrong dimension, RGB?/Grayscale)

[andife]

Hello,

My objective is to optimize and calibrate my model for INT8 usage.

After saving the model from tensorflow2 to the saved_model format,
the input dimension is (None, 128,128,1). At the import step at the workbench,
I set the dimensions to (1,128,128,1) for batch 1 and I set the image colorspace
to grayscale. Processing with 11 files in the standard mode works fine
an gives measurement results.

My test files for "Segmantic Segmentation" are in the "Common Semantic Segmentation" format.
Can I specify for int8-Calibration that the jpg-Files are imported as grayscale?

The jpg-files in css-format are grayscale (only 1 component)

"file pic1.jpg: JPEG image data, JFIF standard 1.01, aspect ratio, density 1x1, segment length 16, baseline, precision 8, 640x480, components 1"

After that I tried to optimize with tpot (online and a local linux). There I got the following error message

The -result-archive argument is not set. The default value /home/workbench/.workbench/models/2/21/job_artifacts/artifact.tar.gz will be used.

[setupvars.sh] OpenVINO environment initialized


0%| |00:00

0%| |00:00
4%|▍ |00:00Traceback (most recent call last):

File "/usr/local/bin/pot", line 11, in <module>

load_entry_point('pot==1.0', 'console_scripts', 'pot')()

File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/app/run.py", line 36, in main

app(sys.argv[1:])

File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/app/run.py", line 60, in app

metrics = optimize(config)

File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/app/run.py", line 138, in optimize

compressed_model = pipeline.run(model)

File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/compression/pipeline/pipeline.py", line 54, in run

result = self.collect_statistics_and_run(model, current_algo_seq)

File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/compression/pipeline/pipeline.py", line 64, in collect_statistics_and_run

model = algo.run(model)

File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/compression/algorithms/quantization/default/algorithm.py", line 94, in run

self.algorithms[1].algo_collector.compute_statistics(model)

File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/compression/statistics/collector.py", line 70, in compute_statistics

_, stats_ = self._engine.predict(combined_stats, sampler)

File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/compression/engines/ac_engine.py", line 166, in predict

stdout_redirect(self._model_evaluator.process_dataset_async, **args)

File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/compression/utils/logger.py", line 129, in stdout_redirect

res = fn(*args, **kwargs)

File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/libs/open_model_zoo/tools/accuracy_checker/accuracy_checker/evaluators/quantization_model_evaluator.py", line 142, in process_dataset_async

self._fill_free_irs(free_irs, queued_irs, infer_requests_pool, dataset_iterator, **kwargs)

File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/libs/open_model_zoo/tools/accuracy_checker/accuracy_checker/evaluators/quantization_model_evaluator.py", line 329, in _fill_free_irs

batch_input, batch_meta = self._get_batch_input(batch_inputs, batch_annotation)

File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/libs/open_model_zoo/tools/accuracy_checker/accuracy_checker/evaluators/quantization_model_evaluator.py", line 80, in _get_batch_input

filled_inputs = self.input_feeder.fill_inputs(batch_input)

File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/libs/open_model_zoo/tools/accuracy_checker/accuracy_checker/launcher/input_feeder.py", line 193, in fill_inputs

inputs = self.fill_non_constant_inputs(data_representation_batch)

File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/libs/open_model_zoo/tools/accuracy_checker/accuracy_checker/launcher/input_feeder.py", line 187, in fill_non_constant_inputs

filled_inputs, extract_image_representations(data_representation_batch, meta_only=True)

File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/libs/open_model_zoo/tools/accuracy_checker/accuracy_checker/launcher/input_feeder.py", line 324, in _transform_batch

self.precision_mapping.get(layer_name)

File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/libs/open_model_zoo/tools/accuracy_checker/accuracy_checker/launcher/dlsdk_launcher.py", line 840, in fit_to_input

return self._align_data_shape(data, layer_name, layout)

File "/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/libs/open_model_zoo/tools/accuracy_checker/accuracy_checker/launcher/dlsdk_launcher.py", line 606, in _align_data_shape

return data.reshape(input_shape) if not self.disable_resize_to_input else data

ValueError: cannot reshape array of size 49152 into shape (1,1,128,128)

=> As 49152/128/128 = 3, it seems that the dimension is somehow set to 3? RGB?

Any hints, which settings I've to check?

Thank you

[Zulkifli_Intel]

Hello Andreas Fehlner,

Thank you for reaching out to us.

Please share with us your model and the dataset so that we can investigate this issue further.

Sincerely,

Zul

[andife]

Hello,

I've attached the requested data.

Thank you

Andreas

[andife]

Hello,

I tried setting up a correct configuration file for the  accuracy checker.

(I also tried out

a)

reader:

  type: opencv_imread
 reading_flag: gray

b)

#preprocessing:
- type: bgr_to_gray

=> I think this should not be correct as the original image only contains one channel

My best guess is attached:

The error log is:

andi@er:~/workT/currproj$ accuracy_check -c accuracy_css_gray2.yml
['models']
Processing info:
model: moba
launcher: dlsdk
device: CPU
dataset: d500
OpenCV version: 4.5.3-openvino
Annotation conversion for d500 dataset has been started
Parameters to be used for conversion:
converter: common_semantic_segmentation
images_dir: /home/andi/workT/currproj/css/Images
masks_dir: /home/andi/workT/currproj/css/Masks
image_postfix: .png
mask_postfix: .png
dataset_meta_file: /home/andi/workT/currproj/css/dataset_meta.json
Annotation conversion for d500 dataset has been finished
IE version: 2021.4.0-3839-cd81789d294-releases/2021/4
Loaded CPU plugin version:
CPU - MKLDNNPlugin: 2.1.2021.4.0-3839-cd81789d294-releases/2021/4
Found model /home/andi/workT/currproj/saved_model.xml
Found weights /home/andi/workT/currproj/saved_model.bin
Input info:
Layer name: input_1
precision: FP32
shape [1, 1, 128, 128]

Output info
Layer name: StatefulPartitionedCall/model/conv2d_25/BiasAdd/Softmax
precision: FP32
shape: [1, 3, 128, 128]

0%| | 0/495 [00:00<?]23:14:46 accuracy_checker ERROR: cannot reshape array of size 49152 into shape (1,1,128,128)
Traceback (most recent call last):
File "/opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/main.py", line 419, in main
evaluator.process_dataset(
File "/opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/evaluators/model_evaluator.py", line 334, in process_dataset_sync
filled_inputs, batch_meta = self._get_batch_input(batch_annotation, batch_input)
File "/opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/evaluators/model_evaluator.py", line 235, in _get_batch_input
filled_inputs = self.input_feeder.fill_inputs(batch_input)
File "/opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/launcher/input_feeder.py", line 193, in fill_inputs
inputs = self.fill_non_constant_inputs(data_representation_batch)
File "/opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/launcher/input_feeder.py", line 186, in fill_non_constant_inputs
return self._transform_batch(
File "/opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/launcher/input_feeder.py", line 321, in _transform_batch
batch_data[layer_name] = self.input_transform_func(
File "/opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/launcher/dlsdk_launcher.py", line 840, in fit_to_input
return self._align_data_shape(data, layer_name, layout)
File "/opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/launcher/dlsdk_launcher.py", line 606, in _align_data_shape
return data.reshape(input_shape) if not self.disable_resize_to_input else data
ValueError: cannot reshape array of size 49152 into shape (1,1,128,128)

[Zulkifli_Intel]

Hello Andreas Fehlner,

Thank you for sharing the model and the dataset with us. Can we know where is the source of your model and which TensorFlow 2 model that you are using (eg: EfficientDet, ssd_mobilenet, etc)?

Sincerely,

Zul

[andife]

Hello Zul,

thank you for the answer!

The original model (a unet/sdunet) was custom made in tensorflow 2.4.1/2.4.2 and saved as 'SavedModel'.

The variant attached here is a very reduced and not trained version, which shows the same behavoir as the original model.
The input images are grayscale.

Sincerely,

Andreas

[Zulkifli_Intel]

Hello Andreas Fehlner,

We can optimize the model to INT8 and run the accuracy test. The current workaround is you need to change the preprocessing to bgr_to_gray, in order for it to work without error. 

Sincerely,

Zul

[andife]

Hello Zul,

thank you for investigating the issue. This really helped a lot.
I also realized that the missing postprocessing part in my yml-file produced unexpected behavoir.

At the moment different issues exist (running workbench with openvino 2021.4):

1) No accuracy values although an annotated file is used (Common semantic segmentation)
2) Although I've done int8 optimization, all slices remained FP32.

Thank you

Andreas

My steps were the following:

 

 

 

 

 

 

  

[Zulkifli_Intel]

Hello Andreas Fehlner,

Thank you for sharing your steps. To run the accuracy checker and convert to INT8, please follow these steps:

Run the Accuracy Check:

-      Click on Configure Accuracy

-      Change usage to semantic segmentation

-      Run Accuracy Check

If it results in “ValueError: cannot reshape array of size 49152 into shape (1,1,128,128)”

Reconfigure the accuracy checker setting by:

-      Click on configure accuracy

-      Change from Basic to Advanced mood

      

-      modify the preprocessing:

•   type: bgr_to_gray

-      Run Accuracy Check

Optimize to INT8 precision

-      Click on Perform

-      INT8

-      Optimize

-      Select the annotated dataset

-      Optimize

Run the Accuracy Check:

-      Click on Configure Accuracy

-      Change from Basic to Advanced mood

-      modify the preprocessing:

•  type: bgr_to_gray

-      Run Accuracy Check

Sincerely,

Zulkifli

[Zulkifli_Intel]

Hello Andreas Fehlner,

This thread will no longer be monitored since we have provided a solution. If you need any additional information from Intel, please submit a new question.

Sincerely,

Zulkifli