Example Code: Incorrect results when switching Precision

[nano]

I'm having a python model and a c++ model with openvino side-by side and compare the CNN-outputs. This works great, when I use the example-code loading the example image using Precision::U8 , load it to the model and compare the CNN-Output - they match exactly.  However, I need the input image to be loaded as Precision::FP32 because I want to preprocess the images myself. (Because the image is in range 0-255 and I want to use imagenet-preprocessing values)

So I use opencv's mat.convertTo(mat, CV_32FC3) to get the image into the target format. When doing this, the result from python and openvino do not match at all, while the did match when not converting to 32FC3.

I think this is due to an issue on how opencv handles it's matrices, which is not compatible to the openvino example!

You can check this minimal example. I convert an U8 image to FP32 and display its values.  You can see clearly, that even though I convert the matrix to FP32, the ".data[id]" still does not return a floating point 32. (I also tried to cast the value to a float and divide after it!)

#include <opencv2/opencv.hpp>

using namespace cv;

int main() {
	
	// converting a u8 mat to 32fc3
	cv::Mat mat = cv::imread("...\\my_image.jpg");  // windows notation
	cv::Mat cvt;

	mat.convertTo(cvt, CV_32FC3);
	cvt = cvt / 255;

	size_t size = 1280 * 1280 * 3;

	std::shared_ptr<float> _data;
	_data.reset(new float[size], std::default_delete<float[]>());

	for (size_t id = 0; id < size; ++id) {

		if (id == 0) {
			std::cout << "For 0: " << cvt.data[id] << std::endl;
		}

		_data.get()[id] = static_cast<float>(cvt.data[id]);
	}

	cv::Mat data_m1(1280, 1280, CV_32FC3, (float*)_data.get());

	cv::imshow("cvt", cvt);  // cvt does contain useful data
	cv::imshow("data_m1", data_m1);  // Note that data_m1 DOES NOT CONTAIN USEFUL DATA

	cv::waitKey(0);
	cv::destroyAllWindows();

	std::cout << "Shape of mat before" << cvt.size().width << ", " << cvt.size().height << ", " << cvt.channels() << std::endl;
	std::cout << "Value at 0 before switch: " << cvt.at<float>(0, 0) << std::endl;

	std::cout << "Shape of mat after" << data_m1.size().width << ", " << data_m1.size().height << ", " << cvt.channels() << std::endl;
	std::cout << "Value at 0 after switch: " << data_m1.at<float>(0, 0) << std::endl;

}

So how do I:

A) Apply ImageNet-Value-Preprocessing automatically? (Normalization is applied on pixel-intensity-values (0...1) , which is not the case when loading images as U8 using opencv (0...255))

B) Load an Image as FP32 to my model?

[Vladimir_Dudnik]

@nano OpenCV cv::Mat class defines data member as uchar* data, so in your example at line 22 integer value will be printed

std::cout << "For 0: " << cvt.data[id] << std::endl;

If your cv::Mat is FP32 type then you would need to access pixels in the way like this (see Accessing pixel image c++ in OpenCV FAQ)

float pixelValue = ExampleMat.at<float>(i,j); 

[Zulkifli_Intel]

Hello Dennis Hugle,

This thread will no longer be monitored since we have provided a solution. If you need any additional information from Intel, please submit a new question.

Sincerely,

Zulkifli