Cannot infer shapes for node "cast3"

[Gouveia__César]

I have a MXNet SqueezeDet model which I tried to convert to an OpenVINO IR. Basically I just run this command:

python deployment_tools\model_optimizer\mo_mxnet.py --input_model squeezedet-0000.params --input_shape (1,3,320,320) --output_dir squeeze_detector --scale 255 --log_level DEBUG

Which gives the following errors:

[ 2020-01-09 11:22:07,468 ] [ DEBUG ] [ infer:129 ]  --------------------
[ 2020-01-09 11:22:07,468 ] [ DEBUG ] [ infer:130 ]  Partial infer for slice_axis25
[ 2020-01-09 11:22:07,469 ] [ DEBUG ] [ infer:131 ]  Op: slice_axis
[ 2020-01-09 11:22:07,469 ] [ DEBUG ] [ infer:142 ]  Inputs:
[ 2020-01-09 11:22:07,469 ] [ DEBUG ] [ infer:32 ]  input[0]: shape = [  1 300   4], value = <UNKNOWN>
[ 2020-01-09 11:22:07,469 ] [ DEBUG ] [ infer:144 ]  Outputs:
[ 2020-01-09 11:22:07,470 ] [ DEBUG ] [ infer:32 ]  output[0]: shape = [  1 300   1], value = <UNKNOWN>
[ 2020-01-09 11:22:07,470 ] [ DEBUG ] [ infer:129 ]  --------------------
[ 2020-01-09 11:22:07,470 ] [ DEBUG ] [ infer:130 ]  Partial infer for reshape83
[ 2020-01-09 11:22:07,470 ] [ DEBUG ] [ infer:131 ]  Op: Reshape
[ 2020-01-09 11:22:07,471 ] [ DEBUG ] [ infer:142 ]  Inputs:
[ 2020-01-09 11:22:07,471 ] [ DEBUG ] [ infer:32 ]  input[0]: shape = [  1 300   1], value = <UNKNOWN>
[ 2020-01-09 11:22:07,471 ] [ DEBUG ] [ infer:32 ]  input[1]: shape = [3], value = [ -1 300   1]
[ 2020-01-09 11:22:07,471 ] [ DEBUG ] [ infer:144 ]  Outputs:
[ 2020-01-09 11:22:07,472 ] [ DEBUG ] [ infer:32 ]  output[0]: shape = [  1 300   1], value = <UNKNOWN>
[ 2020-01-09 11:22:07,472 ] [ DEBUG ] [ infer:129 ]  --------------------
[ 2020-01-09 11:22:07,472 ] [ DEBUG ] [ infer:130 ]  Partial infer for _minusscalar3/lin_/add_
[ 2020-01-09 11:22:07,472 ] [ DEBUG ] [ infer:131 ]  Op: Add
[ 2020-01-09 11:22:07,473 ] [ DEBUG ] [ infer:142 ]  Inputs:
[ 2020-01-09 11:22:07,473 ] [ DEBUG ] [ infer:32 ]  input[0]: shape = [1], value = [-1.]
[ 2020-01-09 11:22:07,474 ] [ DEBUG ] [ infer:32 ]  input[1]: shape = [  1 300   1], value = <UNKNOWN>
[ 2020-01-09 11:22:07,474 ] [ DEBUG ] [ infer:144 ]  Outputs:
[ 2020-01-09 11:22:07,475 ] [ DEBUG ] [ infer:32 ]  output[0]: shape = [  1 300   1], value = <UNKNOWN>
[ 2020-01-09 11:22:07,475 ] [ DEBUG ] [ infer:129 ]  --------------------
[ 2020-01-09 11:22:07,475 ] [ DEBUG ] [ infer:130 ]  Partial infer for _plusscalar71/lin_
[ 2020-01-09 11:22:07,475 ] [ DEBUG ] [ infer:131 ]  Op: Add
[ 2020-01-09 11:22:07,476 ] [ DEBUG ] [ infer:142 ]  Inputs:
[ 2020-01-09 11:22:07,476 ] [ DEBUG ] [ infer:32 ]  input[0]: shape = [  1 300   1], value = <UNKNOWN>
[ 2020-01-09 11:22:07,476 ] [ DEBUG ] [ infer:32 ]  input[1]: shape = [1], value = [1.]
[ 2020-01-09 11:22:07,477 ] [ DEBUG ] [ infer:144 ]  Outputs:
[ 2020-01-09 11:22:07,481 ] [ DEBUG ] [ infer:32 ]  output[0]: shape = [  1 300   1], value = <UNKNOWN>
[ 2020-01-09 11:22:07,482 ] [ DEBUG ] [ infer:129 ]  --------------------
[ 2020-01-09 11:22:07,482 ] [ DEBUG ] [ infer:130 ]  Partial infer for _mulscalar71/lin_
[ 2020-01-09 11:22:07,482 ] [ DEBUG ] [ infer:131 ]  Op: Mul
[ 2020-01-09 11:22:07,482 ] [ DEBUG ] [ infer:142 ]  Inputs:
[ 2020-01-09 11:22:07,483 ] [ DEBUG ] [ infer:32 ]  input[0]: shape = [  1 300   1], value = <UNKNOWN>
[ 2020-01-09 11:22:07,483 ] [ DEBUG ] [ infer:32 ]  input[1]: shape = [1], value = [2.7182817]
[ 2020-01-09 11:22:07,483 ] [ DEBUG ] [ infer:144 ]  Outputs:
[ 2020-01-09 11:22:07,484 ] [ DEBUG ] [ infer:32 ]  output[0]: shape = [  1 300   1], value = <UNKNOWN>
[ 2020-01-09 11:22:07,484 ] [ DEBUG ] [ infer:129 ]  --------------------
[ 2020-01-09 11:22:07,484 ] [ DEBUG ] [ infer:130 ]  Partial infer for _greater_scalar3/lin_
[ 2020-01-09 11:22:07,484 ] [ DEBUG ] [ infer:131 ]  Op: Greater
[ 2020-01-09 11:22:07,484 ] [ DEBUG ] [ infer:142 ]  Inputs:
[ 2020-01-09 11:22:07,485 ] [ DEBUG ] [ infer:32 ]  input[0]: shape = [  1 300   1], value = <UNKNOWN>
[ 2020-01-09 11:22:07,485 ] [ DEBUG ] [ infer:32 ]  input[1]: shape = [1], value = [1.]
[ 2020-01-09 11:22:07,485 ] [ DEBUG ] [ infer:144 ]  Outputs:
[ 2020-01-09 11:22:07,485 ] [ DEBUG ] [ infer:32 ]  output[0]: shape = [  1 300   1], value = <UNKNOWN>
[ 2020-01-09 11:22:07,486 ] [ DEBUG ] [ infer:129 ]  --------------------
[ 2020-01-09 11:22:07,486 ] [ DEBUG ] [ infer:130 ]  Partial infer for cast3
[ 2020-01-09 11:22:07,486 ] [ DEBUG ] [ infer:131 ]  Op: Cast
[ ERROR ]  Cannot infer shapes or values for node "cast3".
[ ERROR ]  Destination type of "Cast" operation should be extracted earlier
[ ERROR ]
[ ERROR ]  It can happen due to bug in custom shape infer function <function Cast.infer at 0x0000019FC6DB9048>.
[ ERROR ]  Or because the node inputs have incorrect values/shapes.
[ ERROR ]  Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
[ 2020-01-09 11:22:07,487 ] [ DEBUG ] [ infer:196 ]  Node "cast3" attributes: {'symbol_dict': {'op': 'Cast', 'name': 'cast3', 'attrs': {'dtype': 'float32'}, 'inputs': [[246, 0, 0]]}, '_in_ports': {0: {'control_flow': False}}, '_out_ports': {0: {'control_flow': False}}, 'kind': 'op', 'name': 'cast3', 'type': 'Convert', 'op': 'Cast', 'infer': <function Cast.infer at 0x0000019FC6DB9048>, 'precision': 'FP32', 'type_infer': <function Cast.type_infer at 0x0000019FC6DA7F28>, 'dst_type': None, 'in_ports_count': 1, 'out_ports_count': 1, 'dim_attrs': ['spatial_dims', 'batch_dims', 'axis', 'channel_dims'], 'shape_attrs': ['pad', 'shape', 'output_shape', 'window', 'stride'], 'IE': [('layer', [('id', <function Op.substitute_ie_attrs.<locals>.<lambda> at 0x0000019FC848BD90>), 'name', 'precision', 'type'], [('data', [('precision', <function Cast.backend_attrs.<locals>.<lambda> at 0x0000019FC848BEA0>)], []), '@ports', '@consts'])], 'is_output_reachable': True, 'is_undead': False, 'is_const_producer': False, 'is_partial_inferred': False}
[ ERROR ]  Destination type of "Cast" operation should be extracted earlier
Stopped shape/value propagation at "cast3" node.
 For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #38.
Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Destination type of "Cast" operation should be extracted earlier
Stopped shape/value propagation at "cast3" node.
 For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #38.
[ 2020-01-09 11:22:07,489 ] [ DEBUG ] [ main:304 ]  Traceback (most recent call last):
  File "C:\Program Files (x86)\IntelSWTools\openvino_2019.3.379\deployment_tools\model_optimizer\mo\middle\passes\infer.py", line 132, in partial_infer
    node.infer(node)
  File "C:\Program Files (x86)\IntelSWTools\openvino_2019.3.379\deployment_tools\model_optimizer\extensions\ops\Cast.py", line 50, in infer
    assert node.has_valid('dst_type'), 'Destination type of "Cast" operation should be extracted earlier'
AssertionError: Destination type of "Cast" operation should be extracted earlier

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Program Files (x86)\IntelSWTools\openvino_2019.3.379\deployment_tools\model_optimizer\mo\utils\class_registration.py", line 273, in apply_replacements
    for_graph_and_each_sub_graph_recursively(graph, replacer.find_and_replace_pattern)
  File "C:\Program Files (x86)\IntelSWTools\openvino_2019.3.379\deployment_tools\model_optimizer\mo\middle\pattern_match.py", line 58, in for_graph_and_each_sub_graph_recursively
    func(graph)
  File "C:\Program Files (x86)\IntelSWTools\openvino_2019.3.379\deployment_tools\model_optimizer\extensions\middle\PartialInfer.py", line 31, in find_and_replace_pattern
    partial_infer(graph)
  File "C:\Program Files (x86)\IntelSWTools\openvino_2019.3.379\deployment_tools\model_optimizer\mo\middle\passes\infer.py", line 198, in partial_infer
    refer_to_faq_msg(38)) from err
mo.utils.error.Error: Destination type of "Cast" operation should be extracted earlier
Stopped shape/value propagation at "cast3" node.
 For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #38.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Program Files (x86)\IntelSWTools\openvino_2019.3.379\deployment_tools\model_optimizer\mo\main.py", line 298, in main
    return driver(argv)
  File "C:\Program Files (x86)\IntelSWTools\openvino_2019.3.379\deployment_tools\model_optimizer\mo\main.py", line 263, in driver
    ret_res = mo_mxnet.driver(argv, argv.input_model, model_name, argv.output_dir)
  File "C:\Program Files (x86)\IntelSWTools\openvino_2019.3.379\deployment_tools\model_optimizer\mo\pipeline\mx.py", line 96, in driver
    class_registration.apply_replacements(graph, class_registration.ClassType.MIDDLE_REPLACER)
  File "C:\Program Files (x86)\IntelSWTools\openvino_2019.3.379\deployment_tools\model_optimizer\mo\utils\class_registration.py", line 293, in apply_replacements
    )) from err
mo.utils.error.Error: Destination type of "Cast" operation should be extracted earlier
Stopped shape/value propagation at "cast3" node.
 For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #38.
Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Destination type of "Cast" operation should be extracted earlier
Stopped shape/value propagation at "cast3" node.
 For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #38.

Has someone already deal with this cast layer problem? Any solutions? I don't see the a cast layer on the supported layers on MXNet, however, if it was a non-supported layer, the debug messages should have been around that and not because of the shapes.

If you need additional information (model json, etc...) please send a PM or ask here in this thread.

Thanks,

César.

[Gouveia__César]

So I solved this bug related to cast, I was just passing the wrong denomination for the precision. Now the model converts correctly, however, now another problem arises. When I run the inference engine I have some slice_axis layers (MXNet json) which get converted to Crop type (OpenVINO xml). This is a problem because it gives the following error:

(vb36) C:\Program Files (x86)\IntelSWTools\openvino_2019.3.379>"<USERPROFILE>\Documents\Intel\OpenVINO\inference_engine_samples_build\intel64\Release\classification_sample_async.exe" -i data\detect.jpg -m squeezedet-0000.xml -d CPU
[ INFO ] InferenceEngine:
        API version ............ 2.1
        Build .................. 32974
        Description ....... API
[ INFO ] Parsing input parameters
[ INFO ] Parsing input parameters
[ INFO ] Files were added: 1
[ INFO ]     data\detect.jpg
[ INFO ] Creating Inference Engine
        CPU
        MKLDNNPlugin version ......... 2.1
        Build ........... 32974

[ INFO ] Loading network files
[ INFO ] Preparing input blobs
[ INFO ] Batch size is 1
[ INFO ] Loading model to the device
[ ERROR ] Crop supports only 2d, 4d and 5d blobs.

What is the reason for Crop not having support for 3d blobs? is this a bug? or there is a specific reason for that?

[Gouveia__César]

Can someone address this problem?

Thanks, César.

[Luis_at_Intel]

Hi Cesar,

Thanks for reaching out. The issue is very clear based on the error, but I don't have an explanation as to why 3d blobs are not supported under Crop.

May I ask if its possible to share your MXNet model, command used to convert to IR, an input image for testing and if you made any modifications to the classification_sample_async. We would like to reproduce the issue if possible, please send me a PM with the info in case you don't want to share it publicly.

Regards,

Luis

[Gouveia__César · Accepted solution]

Hi Luis,

I have found a workaround, which was to reshape (in Keras) the inputs given to the slice (Crops in OpenVINO) to 4d instead of 3d, basically adding another dimension at the left (if you have a (3,32,32) blob, after reshaping you should have (1,3,32,32). And this has solved the problem in my case. However I still don't understand why 3d blobs are not supported for the Crop function on OpenVINO. If you want to recreate the issue, the model used was SqueezeDet which can be found on the modelzoo.

Thank you very much for your reply!

[Luis_at_Intel]

Thanks for sharing the information and your workaround. I will try to find out the reason behind Crop only supporting 2d, 4d, and 5d blobs, I'll get back to you as soon as I have something to share.

Regards,

Luis