Is it possible to implement batch inference in OpenCV using OpenVINO backend?

[Shelke__Sagar]

Hi,

I am using OpenCV DNN module for NN inference. Inference on single image, i.e. batch size of 1 works fine but when I try to use more than one images in inference batch, it fails giving error

what(): OpenCV(4.0.1-openvino) /home/jenkins/workspace/OpenCV/OpenVINO/build/opencv/modules/dnn/src/op_inf_engine.cpp:553: error: (-215:Assertion failed) Failed to initialize Inference Engine backend: Input blob size is not equal network input size (1350000!=270000). in function 'initPlugin'

I am using blobFromImages instead of blobFromImage. Blob has 'batch_size' images in it but just inference doesn't work.

I know OpenVINO API has SetBatch() method.

Should I instead switch to OpenVINO API for inference?

Thanks for your time.

[Shubha_R_Intel]

Dear Sagar,

First I need to ask why are you using OpenCV DNN module for NN inference ? I mean, sure, it's allowed but just wondering if you had a particular reason for doing that rather than using Inference Engine for inference.

We also have the new Dynamic Batching feature. You can read about it here:

Dynamic Batching

Please report back on this forum and hopefully I can address your questions.

Thanks,

Shubha

[Shelke__Sagar]

Hi Shubha,

It is legacy code and we are updating it to support the batch inference. OpenCV takes care of a few things like loading plugins. Inference Engine API does have lots of cool features like dynamic batching and there is no problem in using it directly. I was just wondering whether the batch inference is supported through OpenCV or not!

If possible, Can you point me in the direction where I can find a classification example using Inference Engine API in c++? 

Thanks,

Sagar

[Shubha_R_Intel]

Dearest Sagar,

OpenCV is outside of the purview of this forum, which focuses on Model Optimizer and Inference Engine.  Please see my answers to these forum posters who also asked an OpenCV question:

https://software.intel.com/en-us/forums/computer-vision/topic/807397

https://software.intel.com/en-us/forums/computer-vision/topic/806937

Here is what I said on those forum posts:

Dear Customer,

At first glance, this doesn't seem to be an OpenVino problem.

You may get faster response if you posted here:

http://answers.opencv.org/questions/

In the meantime, it would be helpful to know if you can reproduce this issue in the standard github version of OpenCV:

https://github.com/opencv/opencv.git

Thanks for using OpenVino !

Shubha

[Shelke__Sagar]

Hi Shubha,

I was able to solve this problem. Since batch size setting option is not available in OpenCV, you can do either of two things

1. Compile model with --batch parameter set to desired batch size while using OpenVINO model optimizer.

2. While giving input shape, consider batch size. Normal input for SSD 300 will be [1, 300, 300, 3] but with batch size N, it will be [N, 300, 300, 3].

But this poses restriction unlike OpenVINO because every time we change batch size, the model should be recompiled. 

Thanks for your help.

-Sagar