Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.

[VPU] Permute has to provide order dimension 4

ha__minh_quyet
Beginner
977 Views

Dear all,

I converted my tensorflow model into IR format. However, I got the error . RuntimeError: [VPU] Permute has to provide order dimension 4 (I use the Python version).

I use MYRIAD to inference: python inference.py -m model/ir_format/asr_horovod_ir_test.xml -d MYRIAD -i model/27407_004.wav.

How can I fix the error? All my own layer use Permute have 3 dimensions instead of  4.

Best regards

0 Kudos
13 Replies
Shubha_R_Intel
Employee
977 Views

Hi quyet. Are you perhaps using an unsupported layer in your model ?

Kindly check this list :

https://movidius.github.io/ncsdk/release_notes.html

0 Kudos
ha__minh_quyet
Beginner
977 Views

Dear Shubha,

Thank you for your response,

I will check the list that you recommended. Actually, The issue come form Squeeze layer. It use Permute to transpose tensors. It convert 1 tensor with 4 dimension to 3 tensor with 3 dimension. The IR format for the Squeeze layer is as following:

<layer id="43" name="lambda_1/Squeeze" precision="FP16" type="Reshape">

      <data dim="1,96,240"/>

      <input>

                 <port id="0">

                          <dim>1</dim>

                          <dim>96</dim>

                          <dim>240</dim>

                          <dim>1</dim>

                 </port>

      </input>

      <output>

                <port id="1">

                         <dim>1</dim>

                         <dim>96</dim>

                        <dim>240</dim>

                </port>

         </output>

</layer>

<layer id="44" name="lambda_1/transpose" precision="FP16" type="Permute">

       <data order="2,0,1"/>

       <input>

                <port id="0">

                       <dim>1</dim>

                       <dim>96</dim>

                       <dim>240</dim>

                </port>

        </input>

        <output>

                <port id="1">

                        <dim>240</dim>

                        <dim>1</dim>

                        <dim>96</dim>

                </port>

         </output>

</layer>

 

Thank you so much for your support.

Best regards,

Minh Quyet

0 Kudos
Shubha_R_Intel
Employee
977 Views

Dearest quyet,

congratulations you may have found a bug ! Can you kindly attach your model as a zip file as well as the full model optimizer command you used so that I can reproduce it and file a bug ticket ?

In meantime you can manually edit the IR (xml) to change order of Reshape and Permute – do Permute in 4D first and then Reshape to 3D. 

Thanks for your patience !

Shubha

0 Kudos
ha__minh_quyet
Beginner
977 Views

Shubha R. (Intel) wrote:

Dearest quyet,

congratulations you may have found a bug ! Can you kindly attach your model as a zip file as well as the full model optimizer command you used so that I can reproduce it and file a bug ticket ?

In meantime you can manually edit the IR (xml) to change order of Reshape and Permute – do Permute in 4D first and then Reshape to 3D. 

Thanks for your patience !

Shubha

Dear Shubha,

Thank you very much for your comment. The model belongs to work, so I can not upload here. Is there a private way I can send it to you? In addition, I will manually modify the IR (xml) to add a Reshape layer as your advise.

Thank you so much for your help. You save my days. I am looking forward to hearing from you soon.

Best regards,

HA Minh Quyet

0 Kudos
Shubha_R_Intel
Employee
977 Views

It's ok quyet. If it belongs to your work, then please don't send it to us. We'll try to reproduce the bug in our own way.

Have a wonderful day and thanks for using OpenVino !

Shubha

0 Kudos
ha__minh_quyet
Beginner
977 Views

Dear Shubha,

Sorry for the inconvenience. I have a question about the command PermuteAttrs.create_permute_attrs(node, attrs=[('dim', 'output:0')]). My issue start from Squeeze layer. As your advise, i will add reshape layer before and after Permute layer. However, If i manually edit the IR file, it will affect to the id of layer. Therefore, I try to edit by rewrite the squeeze operation (<INSTALL_PATH>/deployment_tools/model_optimizer/mo/front/common/partial_infer/squeeze.py). But i do not find the command add Permute layer in the python file except the command PermuteAttrs.create_permute_attrs(node, attrs=[('dim', 'output:0')]). And I do not understand where you can add the Permute operation after the Squeeze operation. In my tensorflow graph, therer is not any Permute operation after the Squeeze operation. I searched about PermuteAttrs.create_permute_attrs(node, attrs=[('dim', 'output:0')]) in google but i can not find any documents about it.

Thank you so much for your close support.

Best regards,

Ha Minh Quyet

0 Kudos
ha__minh_quyet
Beginner
977 Views

Dear Shubha,

I manually edited the xml file and figure out the problems with Permute operation in Neural compute stick2. However, I got another error :

[VPU] Cannot convert layer "bidirectional_1/while/LoopCond/TensorIteratorCondition_/TensorIterator" due to unsupported layer type "TensorIterator"

I guess that the TensorIterator layer was unsupported by Inferences Engine in Neural Compute Stick as the document https://docs.openvinotoolkit.org/R5/_docs_IE_DG_supported_plugins_Supported_Devices.html. Can I do something to figure the error or wait the new release of Openvino toolkit?.

In addition, in the document said that the TensorIterator was supported by Inferences Engine in CPU. But, i got the error Unsupported primitive of type: TensorIterator name: bidirectional_1/while/LoopCond/TensorIteratorCondition_/TensorIterator. Is it related to bug of Inferences Engine?

Best regards,

Ha Minh Quyet

0 Kudos
Drakopoulos__Fotis
977 Views

ha, minh quyet wrote:

Dear Shubha,

I manually edited the xml file and figure out the problems with Permute operation in Neural compute stick2. However, I got another error :

[VPU] Cannot convert layer "bidirectional_1/while/LoopCond/TensorIteratorCondition_/TensorIterator" due to unsupported layer type "TensorIterator"

I guess that the TensorIterator layer was unsupported by Inferences Engine in Neural Compute Stick as the document https://docs.openvinotoolkit.org/R5/_docs_IE_DG_supported_plugins_Suppor.... Can I do something to figure the error or wait the new release of Openvino toolkit?.

In addition, in the document said that the TensorIterator was supported by Inferences Engine in CPU. But, i got the error Unsupported primitive of type: TensorIterator name: bidirectional_1/while/LoopCond/TensorIteratorCondition_/TensorIterator. Is it related to bug of Inferences Engine?

Best regards,

Ha Minh Quyet

Hello,

Could you please tell me what did you do exactly to solve the Permute error? I am still trying to solve this so I would really appreciate your feedback.

Best regards,

Fotis

0 Kudos
Shubha_R_Intel
Employee
977 Views

Dear quyet,  PermuteAttrs.create_permute_attrs(node, attrs=[('dim', 'output:0')]). is a function which Model Optimizer uses to define rules for layout conversation to make Inference Engine happy. You are right - you will not find anything related to this in Tensorflow. It's entirely a Model Optimizer-to-Inference Engine function.

I'm terribly sorry that you're having such problems even on CPU. When you get that error Unsupported primitive of type:  it means that the operation does not exist in the model optimizer (and likely it doesn't exist at the Inference Engine too).  At least the Myriad document tells the truth about the "TensorIterator" primitive not being supported.

So first, I have already filed a bug about the original problem for you. Now I need to file another about the CPU documentation being inaccurate. Can you attach the xml file here which has the TensorIterator error ? I will file a bug for you.

I will send the IR xml file to experts within the development team. I will post here when I find out more info.

Thanks so much for your patience !

Thanks,

Shubha

0 Kudos
ha__minh_quyet
Beginner
977 Views

Drakopoulos, Fotis wrote:

Quote:

ha, minh quyet wrote:

 

Dear Shubha,

I manually edited the xml file and figure out the problems with Permute operation in Neural compute stick2. However, I got another error :

[VPU] Cannot convert layer "bidirectional_1/while/LoopCond/TensorIteratorCondition_/TensorIterator" due to unsupported layer type "TensorIterator"

I guess that the TensorIterator layer was unsupported by Inferences Engine in Neural Compute Stick as the document https://docs.openvinotoolkit.org/R5/_docs_IE_DG_supported_plugins_Suppor.... Can I do something to figure the error or wait the new release of Openvino toolkit?.

In addition, in the document said that the TensorIterator was supported by Inferences Engine in CPU. But, i got the error Unsupported primitive of type: TensorIterator name: bidirectional_1/while/LoopCond/TensorIteratorCondition_/TensorIterator. Is it related to bug of Inferences Engine?

Best regards,

Ha Minh Quyet

 

 

Hello,

Could you please tell me what did you do exactly to solve the Permute error? I am still trying to solve this so I would really appreciate your feedback.

Best regards,

Fotis

Dear Fotis,

As the Shubha's advise in the previous comment, we need to add 2 reshape layer before and after the Permute layer. Therefore, I manually edited the xml file as following:

Previous:

<layer id="44" name="lambda_1/transpose" precision="FP16" type="Permute">

       <data order="2,0,1"/>

       <input>

                <port id="0">

                       <dim>1</dim>

                       <dim>96</dim>

                       <dim>240</dim>

                </port>

        </input>

        <output>

                <port id="1">

                        <dim>240</dim>

                        <dim>1</dim>

                        <dim>96</dim>

                </port>

         </output>

</layer>

.......

<edges>

         .......

         <edge from-layer="43" from-port="0" to-layer="44" to-port="0"/> 

         <edge from-layer="44" from-port="0" to-layer="45" to-port="0"/>

         .......

</edges>

 

After:

<layer id="163" name="lambda_1/reshape_before" precision="FP16" type="Reshape">

        <data dim="1,1,96,240"/>

       <input>

                <port id="0">

                       <dim>1</dim>

                       <dim>96</dim>

                       <dim>240</dim>

                </port>

        </input>

        <output>

                <port id="1">

                        <dim>1</dim>

                        <dim>240</dim>

                        <dim>1</dim>

                        <dim>96</dim>

                </port>

         </output>

</layer>

<layer id="44" name="lambda_1/transpose" precision="FP16" type="Permute">

       <data order="0,3,1,2"/>

       <input>

                <port id="0">

                       <dim>1</dim>

                       <dim>1</dim>

                       <dim>96</dim>

                       <dim>240</dim>

                </port>

        </input>

        <output>

                <port id="1">

                        <dim>1</dim>

                        <dim>240</dim>

                        <dim>1</dim>

                        <dim>96</dim>

                </port>

         </output>

</layer>

<layer id="164" name="lambda_1/reshape_after" precision="FP16" type="Reshape">

        <data dim="240,1,96"/>

       <input>

                <port id="0">

                        <dim>1</dim>

                        <dim>240</dim>

                        <dim>1</dim>

                        <dim>96</dim>

                </port>

        </input>

        <output>

                <port id="1">

                        <dim>240</dim>

                        <dim>1</dim>

                        <dim>96</dim>

                </port>

         </output>

</layer>

.......

<edges>

         .......

         <edge from-layer="43" from-port="0" to-layer="163" to-port="0"/> 

         <edge from-layer="163" from-port="0" to-layer="44" to-port="0"/>

         <edge from-layer="44" from-port="3" to-layer="164" to-port="0"/> 

        <edge from-layer="164" from-port="1" to-layer="45" to-port="0"/>

         .......

</edges>

Actually, I can't find the python file that custom layers and add the permute operation it in model_optimizer folder. Therefore, I have to fix the xml file manually. :( . I hope it help your work. If you find another solution for the problem, would you please share it to me? Thank you so much.

Best regards,

Ha Minh Quyet.

0 Kudos
Drakopoulos__Fotis
977 Views

ha, minh quyet wrote:

Quote:

Drakopoulos, Fotis wrote:

 

Quote:

ha, minh quyet wrote:

 

Dear Shubha,

I manually edited the xml file and figure out the problems with Permute operation in Neural compute stick2. However, I got another error :

[VPU] Cannot convert layer "bidirectional_1/while/LoopCond/TensorIteratorCondition_/TensorIterator" due to unsupported layer type "TensorIterator"

I guess that the TensorIterator layer was unsupported by Inferences Engine in Neural Compute Stick as the document https://docs.openvinotoolkit.org/R5/_docs_IE_DG_supported_plugins_Suppor.... Can I do something to figure the error or wait the new release of Openvino toolkit?.

In addition, in the document said that the TensorIterator was supported by Inferences Engine in CPU. But, i got the error Unsupported primitive of type: TensorIterator name: bidirectional_1/while/LoopCond/TensorIteratorCondition_/TensorIterator. Is it related to bug of Inferences Engine?

Best regards,

Ha Minh Quyet

 

 

Hello,

Could you please tell me what did you do exactly to solve the Permute error? I am still trying to solve this so I would really appreciate your feedback.

Best regards,

Fotis

 

 

Dear Fotis,

As the Shubha's advise in the previous comment, we need to add 2 reshape layer before and after the Permute layer. Therefore, I manually edited the xml file as following:

 

Actually, I can't find the python file that custom layers and add the permute operation it in model_optimizer folder. Therefore, I have to fix the xml file manually. :( . I hope it help your work. If you find another solution for the problem, would you please share it to me? Thank you so much.

Best regards,

Ha Minh Quyet.

Thanks a lot! I managed to do it the same way you did and it worked. However, it didn't solve my slow performance issue... In any case, if I find a better solution I'll let you know.

Best regards,

Fotis

0 Kudos
ha__minh_quyet
Beginner
977 Views

Dear all,

The issue has been fixed from the release 2019R1 of OpenVINO toolkit. By using the version, I success converting my TensorFlow model to IR format.

Best regards,

Minh Quyet

0 Kudos
Shubha_R_Intel
Employee
977 Views

Dear Minh Quyet,

Thanks for informing the OpenVino community ! And hopefully the problem remains fixed in the latest and greatest OpenVino release, which is 2019R2.01.

Shubha

0 Kudos
Reply