OpenVINO Inference Engine Network Reshape Issue

[Anjaneya_Srujit_Ram]

Hi,

I am using classification_sample python sample with squeezenet1.1 model. I want to reshape my model as the shape of the input image so that I can run my model with shape of the input image without resizing the image and to run the inference part. I have squeezenet1.1 model with (1*3*224*224) as input shape. Now I want to reshape the model with (1*3*320*240) shape. I am using the net.reshape()  API present in the inference engine.

I am running the API as : net.reshape({input_blob : {1, 3, 320, 240}}). Previous input_blob shape is 1*3*224*224.

When I am running the sample, I got an error as :

net.reshape({input_blob : {1, 3, 320, 240}})

 File "ie_api.pyx", line 527, in openvino.inference_engine.ie_api.IENetwork.reshape

RuntimeError: Failed to infer shapes for Convolution layer (conv1_1) with error: New shapes [320,1,3,240] make Kernels(3x3), Channels(1), Output depth(64), Groups(1) not matching weights size: 576 vs 1728

I didn't understand why inference engine reshape API is reading the new shape as  [320,1,3,240]. Please let me know in which format reshape will read the dictionary of new input shapes. I tried by giving various combinations of {1, 3, 320, 240}, but each and every time new shape is taken as [320,1,3,240]. 

Thank You.

[Shubha_R_Intel]

Dear Ramachandruni, Anjaneya Srujit,

Shape Inference  within Inference Engine is tricky. It doesn't always work.  Smart Classroom Demo code shows how to use the Reshape API, namely :

 C:\Program Files (x86)\IntelSWTools\openvino_2019.2.275\inference_engine\demos\smart_classroom_demo\src\cnn.cpp(135)                  
  C:\Program Files (x86)\IntelSWTools\openvino_2019.2.275\inference_engine\demos\smart_classroom_demo\src\detector.cpp(99) 

The last 2 limitations mentioned in the document may be in play here,

Models with fixed dimensions in the dim attribute of the Reshape layer can't be resized.

Shape inference for Interp layer works for almost all cases, except for Caffe models with fixed width and height parameters (for example, semantic-segmentation-adas-0001).

Your RuntimeError: is definitely odd. 

If you look at C:\Program Files (x86)\IntelSWTools\openvino_2019.2.275\deployment_tools\open_model_zoo\models\public\squeezenet1.1\squeezenet1.1.md  I see this :

The model input is a blob that consists of a single image of 1x3x227x227 in BGR order. The BGR mean values need to be subtracted as follows: [104, 117, 123] before passing the image blob into the network.

So the size is 227x227 not 224x224 as you said.

But anyway, if you look at the smart_demo_classroom code I referred you to above you see stuff like this:

std::map<std::string, SizeVector> input_shapes;
        input_shapes[inputInfo.begin()->first] = input_dims;
        net_reader.getNetwork().reshape(input_shapes);

if (outp_shape != cv::Size())
                    blob_wrapper = blob_wrapper.reshape(1, {outp_shape.height, outp_shape.width});
                blob_wrapper.copyTo(vectors->back());

Notice there  are no hard-coded numbers ! I think using "fixed dimensions" may be your problem.

Hope it helps,

Shubha

[Anjaneya_Srujit_Ram]

Hi, 

Thank you for the response. I tried the network reshape API with c++ samples, it is working fine. When I am running the reshape API in python samples I am facing the above error i.e.

net.reshape({input_blob : {1, 3, 320, 240}}),     #  (any shape {1, 3, x, y})

 File "ie_api.pyx", line 527, in openvino.inference_engine.ie_api.IENetwork.reshape

RuntimeError: Failed to infer shapes for Convolution layer (conv1_1) with error: New shapes [320,1,3,240] make Kernels(3x3), Channels(1), Output depth(64), Groups(1) not matching weights size: 576 vs 1728. 

Please let me know why the reshape python API is not working properly and why it is reading the new shape {1, 3, x, y} in a different manner {x, 1, 3, y}. Is there any issue with Shape Inference python API.

Thank You.

[Shubha_R_Intel]

Dear Ramachandruni, Anjaneya Srujit,

reshape definitely works with Python API.  Please take a look C:\Program Files (x86)\IntelSWTools\openvino_2019.2.275\deployment_tools\inference_engine\samples\python_samples\benchmark_app\benchmark\benchmark.py to see an example of  Python Inference Engine reshape.

Also most of the demos under C:\Program Files (x86)\IntelSWTools\openvino_2019.2.275\inference_engine\demos\python_demos  use reshape Python API. Kindly take a look.

Thanks,

Shubha

[Anjaneya_Srujit_Ram]

Hi,

Thank you for the response. With the reference of benchmark_app python sample, I am able to run my sample, but when I am running the sample on squeezenet1.1 onnx model (1 * 3 * 224 * 224), I tried reshaping the model with 284*284 in place of 224*224. I am facing the issue at the softmax output. When I use the shape of (1 * 3 * 284 * 284), the output has shape of (1 * 1000 * 4 * 4) instead of (1 * 1000*  1 * 1). After debugging I guess that issue was with GlobalAveragePool. GlobalAveragePool is not rescaling the output, maybe due to fixed Kernel shapes. 

My queries are :

1. Does network.reshape will modify the kernel shapes according to the new shape or it will use the default kernel shapes which are present in the static model. 

2. How can we differ GlobalAveragePool to normal AveragePool in a model optimizer XML file? Both the AveragePools are using pool_method="avg".

Thank You,

Srujit.

[Shubha_R_Intel]

Dear Ramachandruni, Anjaneya Srujit,

Yes, according to the Supported Framework Layers Doc ONNX AveragePool and GlobalAveragePool converge to pool_method=avg. There other pooling layers available however, for instance GlobalMaxPool which converges to pool_method=max. The only way to change this is by your modifying Model Optimizer code. And since OpenVino is 100% open sourced, there is nothing stopping you from doing that.

And as aforementioned Models with fixed dimensions in the dim attribute of the Reshape layer can't be resized. 

Also, as the Shape Inference document  mentions Reshape cannot be used on all layers. Many layers are supported, however. It's best to take a look at the document and see how it applies to your situation.

Hope it helps,

Shubha