Custom Model: Output Layer Dimension

[gomes__antonio]

Hello,

I'm trying to implement the Mnist example on NCS with a custom model.

I was able to generate the .xml .bin files sucessfully but when I run the files I get this error:

error: (-215:Assertion failed) Failed to initialize Inference Engine backend: [VPU] Internal error: Output in max_pooling2d_5/MaxPool has incorrect width dimension. Expected: 6 or 6 Actual: 5 in function 'initPlugin'

I did the math and the result of output layer is 5.5 (input = 11 / kernel = 2 / stride = 2). The model approached to 5 but VPU needs 6.

How can I resolve this?

[Shubha_R_Intel]

Dear antonio - how are you accomplishing inferencing ? Do you have your own program ? 

It's difficult to diagnose the issue just based on what you've said. 

So seems like IR rounded 5.5 to 5 but Inference Engine is expecting 6. Can you try manually editing(changing) the 5 to a 6 in your generated XML (for the output layer) and see if it works ? First try that experiment and see if it works. Please post your results here.

Thanks,

Shubha

[gomes__antonio]

Hi,

I trained with Keras the follow NN: Conv1-MaxPool1-Conv2-MaxPool2-Conv3-MaxPool3 and all MaxPool have kernel=2,2 and stride=2,2. I saved the model as .pb file and converted to IR through "mo.py" file.

The files .xml and .bin were generated without problems and for inference I used the Python/OpenCv code from the first example of the Raspbian.

 I changed manually the output from 5 to 6 and as expected I have an error in next layer because output of maxpool is input for next layer. So, I changed manually  all following layers but now I have an error in weights size.

The solution that I found was to retrain NN with kernel=3,3 on second and third maxpool layers in order to get an even number on output.

Before:

2 maxpool layer: input=11 kernel=2 stride=2       output:11-2/2 +1=5.5

3 maxpool layer: input=3 kernel=2 stride=2       output:3-2/2 +1=1.5

After:

2 maxpool layer: input=11 kernel=3 stride=2       output:11-3/2 +1=5

3 maxpool layer: input=3 kernel=3 stride=2       output:3-3/2 +1=1

My question was whether there is any way to solve this in the conversion to IR through a flag or so, because it is not very practical to do the calculations.

[Shubha_R_Intel]

Dear antonio,

You've asked a great question. Please give me some time to investigate this !

I will post my answer here.

Thanks for using OpenVino !

Shubha

[Shubha_R_Intel]

Hi Antonio, I have sent you a PM to enable sending me your original model as a zip file so that I can reproduce your error. 

Thanks,

Shubha

[Shubha_R_Intel]

Also Antonio, you may want to retry your original code on the latest release of OpenVino 2019 R1, which was released just yesterday.

Thanks,

Shubha

[Belykh__Aleksandr1]

Hi Shubha, I have the same issue with tiny yolo model.

Loading[ ERROR ] [VPU] Internal error: Output in detector/yolo-v3-tiny/pool2_5/MaxPool@tile=1/4 has incorrect width dimension. Expected: 25 or 25 Actual: 26
I am using latest OpenVINO release and tiny yolo model trained by myself and converted to tensorflow by mystic123 scripts. Have you investigated the way to resolve such issues?

[Belykh__Aleksandr1]

Hello, Shubha!

I have a very similar issue. I am trying to run tiny yolo v3 inference on the modified object_detection_demo_yolov3_async.
We have the yolov3 tiny model trained on our dataset. Its input shape is 1,832,832,3 and it causes the following error:

[ INFO ] Loading model to the plugin
[ INFO ] +0.0011 Loading[ ERROR ] [VPU] Internal error: Output in detector/yolo-v3-tiny/pool2_5/MaxPool@tile=1/4 has incorrect width dimension. Expected: 25 or 25 Actual: 26
Loaded 0 networks

When I try yolo3 tiny model by Hyodo Katsuya (which input is 416x416) everything goes good (except that it is trained on the very different dataset) - Myriad plugin loads it, then inference goes fine and a network even finds something.

The only difference between two networks is a dimension of every input and output shape, our dimensions are twice of Hyodo's.

Can you assist me in this issue, please? Thank you.

[Belykh__Aleksandr1]

To the moderator: I'm sorry for possible duplication of messages = )

Finally, it looks like a bug!

I am adding my layer definition from the xml file of the model at the end of this message.
As I understand this IR Notation Reference 
auto_pad means that needed padding will be added to the input in order to match the output size after pooling
pads_end="1,1" means that additional zero-value pixel will be added to the right and bottom side of the frame, which leads to the matching, mentioned before (26 pixels scanned by kernel 2 with stride 1 produces 25 pixels, but additional padding pixel compensates this inequality).

However, I still have such error message from both R5 2018 and R1 2019 OpenVINO releases:
[ ERROR ] [VPU] Internal error: Output in detector/yolo-v3-tiny/pool2_5/MaxPool@tile=1/4 has incorrect width dimension. Expected: 25 or 25 Actual: 26

Layer definition:
        <layer id="18" name="detector/yolo-v3-tiny/pool2_5/MaxPool" precision="FP16" type="Pooling">
            <data auto_pad="same_upper" exclude-pad="true" kernel="2,2" pads_begin="0,0" pads_end="1,1" pool-method="max" strides="1,1"/>
            <input>
                <port id="0">
                    <dim>1</dim>
                    <dim>512</dim>
                    <dim>26</dim>
                    <dim>26</dim>
                </port>
            </input>
            <output>
                <port id="1">
                    <dim>1</dim>
                    <dim>512</dim>
                    <dim>26</dim>
                    <dim>26</dim>
                </port>
            </output>
        </layer>

[Shubha_R_Intel]

Dear Belykh, Aleksandr,

I fully believe that this is a bug in the Myriad Plugin. I have filed it on your behalf.

Thanks for your cooperation.

Shubha

[Belykh__Aleksandr1]

Dear Shubha.

Good news is that I have found a workaround for the issue.
I have removed  auto_pad="same_upper" and swapped padding "1,1"/"0,0" between pads_begin and pads_end.
The network works fine with pads_begin="1,1" at the moment.

Here is the full picture of the layer description:

        <layer id="18" name="detector/yolo-v3-tiny/pool2_5/MaxPool" precision="FP16" type="Pooling">
            <data exclude-pad="true" kernel="2,2" pads_begin="1,1" pads_end="0,0" pool-method="max" strides="1,1"/>
            <input>
                <port id="0">
                    <dim>1</dim>
                    <dim>512</dim>
                    <dim>26</dim>
                    <dim>26</dim>
                </port>
            </input>
            <output>
                <port id="1">
                    <dim>1</dim>
                    <dim>512</dim>
                    <dim>26</dim>
                    <dim>26</dim>
                </port>
            </output>
        </layer>

[Shubha_R_Intel]

Dearest Belykh, Aleksandr,

Thank you kindly for sharing your findings with the OpenVino community ! This is really appreciated. I have filed a bug on this issue though, and hopefully it should be fixed in the next release.

Thanks for using OpenVino !

Shubha