pytorch transforms.Compose to ov::preprocess::PrePostProcessor

[Enlin]

I have python code like below, how to convert to C++ and get same result.

I get different result beteen Python and C++ version

===python===

model = core.read_model(model=model_path)

compiled_model = core.compile_model(model=model, device_name='AUTO')

input_layer = model.input(0)

output_layer = compiled_model.output(0)

img = Image.open(IMAGE_PATH)

loader = transforms.Compose([

    transforms.ToTensor(),

    transforms.Normalize([0.5], [0.5]),

])

img_float = loader(img).float()

input_image = np.expand_dims(img_float, 0)

result = compiled_model([input_image])

===C++===

ov::preprocess::PrePostProcessor ppp(model);

ppp.input("input").tensor()
    .set_element_type(ov::element::u8)
    .set_layout("NHWC");
ppp.input().model()
    .set_layout("NCHW");
ppp.input("input").preprocess()
    .convert_element_type(ov::element::f32)
    .mean(0.5)
    .scale(0.5);

model = ppp.build();

ov::CompiledModel compiled_model = core.compile_model(model, device_name);

ov::InferRequest infer_request = compiled_model.create_infer_request();
infer_request.set_input_tensor(input_tensor);

infer_request.infer();

[Aznie_Intel]

Hi Enlin,

Thanks for reaching out.

You may refer to the Integrate OpenVINO with Your Application documentation for the steps to implement OpenVINO™ Runtime inference pipeline in your application. The codes are available for Python, C++, and C.

Regards,

Aznie

[Enlin]

Hi Aznie,

thanks for your reply. I did those steps from your recommand document before.

I can run inference in both python and c++ model. horever my points is I can not get same result from same model and same input between python and c++.

I guess this is cause by ppp, is that totally equal from those code:

loader = transforms.Compose([

    transforms.ToTensor(),

    transforms.Normalize([0.5], [0.5]),

])

ppp.input("input").preprocess()
    .convert_element_type(ov::element::f32)
    .mean(0.5)
    .scale(0.5);

Thanks.

Enlin

[Aznie_Intel]

Hi Enlin,

The incorrect result might be due to incorrect filling of tensor data in the preprocessing part of the C++ code. Ensure the channels when preprocessing the image match the channels of the model. You might miss the normalized part of your code.

 auto input_layer = model.get_method("input");

   auto output_layer = compiled_model.get_method("output");

   torch::transforms::Normalize transform({0.5}, {0.5});

Hope this helps.

Regards,

Aznie

[Aznie_Intel]

Hi Enlin,

This thread will no longer be monitored since we have provided a solution. If you need any additional information from Intel, please submit a new question.

Regards,

Aznie

[Enlin]

Hi Aznie,

thanks for your reply.

I find the different result from c++ and python finally.

the input image is RGB grayscale png, in the pytorch:

====python

img = Image.open(image_path)

loader = transforms.Compose([transforms.ToTensor()])

input_tensor = loader(img)

print(input_tensor.shape)

the shape is (1, 120, 160) , and the Tensor data range is 0~1.0

====C++

I need to chage image to grayscal manually

:

OCVReader::OCVReader(const string& filename) {
img = cv::imread(filename);
if (isGrayscale(img)) {
  cv::cvtColor(img, img, cv::COLOR_BGR2GRAY);
 }

FormatReader::ReaderPtr reader(image_path.c_str());

std::shared_ptr<unsigned char> input_data = reader->getData();

ov::Tensor input_tensor = ov::Tensor(input_type, input_shape, input_data.get());

ov::preprocess::PrePostProcessor ppp(model);

ppp.input("input_1").tensor()
  .set_element_type(ov::element::u8)
  .set_layout("NHWC");

//normalize data range to 1.0~0.0
ppp.input("input_1").preprocess().
convert_element_type(ov::element::f32)
.scale(255);
ppp.input().model()
.set_layout("NCHW");

then I can get same result from python and c++ in case of without additional IPP 

horever, after I add normalize operation

==== python

img = Image.open(image_path)

loader = transforms.Compose([transforms.ToTensor(),

                            transforms.Normalize([0.5], [0.5])])

input_tensor = loader(img)

==== C++

ppp.input("input_1").preprocess().
convert_element_type(ov::element::f32)
.scale(255)
.mean(0.5);

 the result is different.

 is there equally preprocess to pytorch transforms.Normalize([0.5], [0.5])] in C++ PrePostProcessor ?

thanks

Enlin

[Enlin · Accepted solution]

after my test, they are equal totally:

python:

img = Image.open(image_path)
loader = transforms.Compose([transforms.ToTensor(),
                            transforms.Normalize([0.5], [0.5])])
input_tensor = loader(img)

c++

ov::preprocess::PrePostProcessor ppp(model);

ppp.input("input_1").tensor()
.set_element_type(ov::element::u8)
.set_layout("NHWC");
ppp.input("input_1").preprocess().
convert_element_type(ov::element::f32)
.scale(255)
.mean(0.5)
.scale(0.5)
;
ppp.input().model()
.set_layout("NCHW");