Incorrect Results using PrePostProcessor Block of OpenVino and FP16 Quantization

[maffan]

Hello,

I am using the ESANET semantic segmentation model (https://github.com/TUI-NICR/ESANet) with Openvino by converting it to ONNX and OpenVino. The original model was in PyTorch which I converted to both OpenVino FP32 and FP16 models. I utilize Intel Integrated GPU, CPU, and both together (MULTI). I am performing inference on a real-time RGB-D stream.

The results are correct in the following scenarios.

1) I used the model without PrePostProcessor Block (specifically without mean and scale) using the FP16 format. I do the normalization manually with a custom preprocessing code before passing the input to the model. This test is done using CPU, integrated GPU, and Multi.

2) I used the model with PrePostProcessor Block (specifically with mean and scale) using the FP32 format. This test is done with CPU, integrated GPU, and Multi.

3) I used the model with PrePostProcessor Block (specifically with mean and scale) using the FP16 format. This test is done with CPU only

Since latency is a major concern, I want to stack preprocessing (normalization and resizing) within the model using the PrePostProcessor module and use the FP16 format. In other words, I want to use both the FP16 format and GPU to minimize latency. However, this scheme is producing the wrong results at the moment.

The version of OpenVino 2022.3.0, and Python 3.7 is used.

Feel free to ask any other questions that might be relevant.

Thank you

[Iffa_Intel]

Hi,

could you share:

1. Relevant model files (ONNX, IR, etc)
2. Custom code involved (if any)
3. Steps/commands that you use till the point of issue
4. Expected "correct" result (eg: You get latency: 8.35, Expected latency:7.00)

Cordially,

Iffa

[maffan]

Thanks for your response. Is it possible to individually share these things with you instead of posting them publically here?

[Iffa_Intel]

Yes, you could email it to me or share it through private links (eg Google Drive with private access).

Cordially,

Iffa

[maffan]

Hello,

I shared with you the necessary info and files. I look forward to hearing from you soon.

Thank you

[Iffa_Intel]

Hi,

I noticed that you have 2 yml files so I tried both of them (note that I had installed the requirements_jetson too).

However, there are packages not found for both of your yml files.

Is there anything else required to be done beforehand?

 

 

 

Cordially,

Iffa

[maffan]

Hello,

As mentioned in the email, you have to only install  environment.yaml (the other are for running it without openvino and pyrealsense2). If you install both it may bring dependencies conflict.

Can you indicate which packages are not found? I can not see it from the posted images.

Thank you

[Iffa_Intel]

Yes, I understand your notes. However, none of them are working (even solo environment.yml) that's the reason I'm looking for possible alternatives.

Attached is the package not found.

Cordially,

Iffa

[maffan]

Hello,

Can you try removing this package names from the environment.yaml file. I created this using Ubuntu 20.04. It could be system decencies conflicts.

If the environment doesn't work, try creating conda environment from rgbd_segmentation.yaml and install openvino runtime and pyrealsense2 on top. Please let me know if it works.

Looking forward to hearing from you soon.

[Iffa_Intel]

We are looking into this issue.

Thank you for your patience.

Cordially,

Iffa

[Iffa_Intel]

Your custom files requires quite a number of tweaking.

However, I managed to set the custom env with OpenVINO and checked on your model's latency, which seems good.

As for your custom inferencing code, there's also some issue (you may refer attachment).

I'm using Ubuntu 20.04 with Realsense D435 camera which is the same as yours.

There's also some chance that the issue came from your software design itself.

To ease this validation process, could you just share (screenshot,etc):

1.Your results pointing to these (correct prediction):

1) I used the model without PrePostProcessor Block (specifically without mean and scale) using the FP16 format. I do the normalization manually with a custom preprocessing code before passing the input to the model. This test is done using CPU, integrated GPU, and Multi.

2) I used the model with PrePostProcessor Block (specifically with mean and scale) using the FP32 format. This test is done with CPU, integrated GPU, and Multi.

3) I used the model with PrePostProcessor Block (specifically with mean and scale) using the FP16 format. This test is done with CPU only

2.Your result indicating the issue (wrong prediction)

This would help us to scope the specific part of the issue.

Thank you.

Cordially,

Iffa

 

[Iffa_Intel]

Thank you for your question. If you need any additional information from Intel, please submit a new question as Intel is no longer monitoring this thread.

Cordially,

Iffa