Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.

OpenVino image scale & benchmark app

DQader
初学者
1,370 次查看

Hi, 

 

I am using OpenVino to compile my mobilenet (in onnx format) based model to then later deploy on FPGA (CoreDLA).

When running the compiled IR graph using the benchmark_app on CPU or FPGA I get a major accuracy drop as compared to running it in Pytorch (>20% difference). My thought is that I am missing something in the image preprocessing step. 

 

In pytorch below is the preprocessing thats done on the image before feeding to model: 

To preprocess the image before feeding it to the model, we apply the following torch transform. Which means convert the input image from range [0, 255] to [0.0, 1.0] first, and then subtract the Imagenet Mean [0.485, 0.456, 0.406],  and divide by Standard deviation [0.229, 0.224, 0.225] in the order of R, G, B.   

 

  transform = Compose([

        ToTensor(),

        Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),

    ])

 

When running the model optimizer I have tried using mean_values argument along with scale_values for standard deviation. I tried all different variations and even tried completely dropping the scaling and in all cases I see the accuracy hover around the same. 

model optimizer command: 

python3 /opt/intel/openvino/deployment_tools/model_optimizer/mo.py --input_model=./model _256.onnx --mean_values=[0.485,0.456,0.406] --scale_values=[0.229,0.224,0.225] --data_type=FP32

Any thoughts on: 

1. Best way to pre-process the image (maybe add the implementation to benchmark_app?)

2. is there a way to convert the images to a 0,1 range before applying mean and s.d.  

3. Is there something else I am missing?

 

Thank you!

 

 

0 项奖励
3 回复数
IntelSupport
管理员
1,331 次查看

 

Hi Dqader,

Thanks for reaching out to us.

We are working on this and will share the information with you soon.

 

Regards,

Aznie


0 项奖励
IntelSupport
管理员
1,320 次查看

Hi Dgader,

My answer to your query is as below.

 

1. Best way to pre-process the image (maybe add the implementation to benchmark_app?)

 

We would suggest you utilize our Post-Training Optimization Tool where the Accuracy_Checker offers the Preprocessing features. Check out: Preprocessors — OpenVINO™ documentation. You can use aspect_ratio_scale for image rescaling in the accuracy_check configuration file.

Accuracy Checker tools allowed you to change the required setting in order to obtain the highest possible accuracy, you can use this tool to find the best-intended result.

 

Example of accuracy-check.yml for mobilenet-v2 that you can refer to : https://github.com/openvinotoolkit/open_model_zoo/blob/master/models/public/mobilenet-v2/accuracy-check.yml

 

Video on Post-training Optimization Tool (POT): https://www.youtube.com/watch?v=cGQesbWuRhk

 

2. Is there a way to convert the images to a 0,1 range before applying mean and s.d.  

 

By using our Post-training Optimization Tool (POT) and Accuracy Checker (AC), you do not required to convert (unlike in PyTorch transform as mentioned above), further explanation can be found here: Converting a Model to Intermediate Representation (IR) — OpenVINO™ documentation

 

You can directly apply Mean and Standard deviation in the accuracy-check.yml

 

Regards,

Aznie


0 项奖励
IntelSupport
管理员
1,259 次查看

Hi Dgader,

This thread will no longer be monitored since we have provided the answers. If you need any additional information from Intel, please submit a new question.


Regards,

Aznie


0 项奖励
回复