Unable to load IR graph in network

[Gupta__Shubham]

Hello,

I am using openvino version 2021.2.185. I am able to convert a custom tensorflow model to IR using model optimizer with command python3 /opt/intel/openvino/deployment_tools/model_optimizer/mo_tf.py --saved_model_dir prediction_model/ -o openvino_model/ --input_shape [1,512,512,3], but not able to load it.

Error:

terminate called after throwing an instance of 'ngraph::NodeValidationFailure'
what(): Check 'sum_of_splits == data_shape[axis].get_length()' failed at ngraph/core/src/op/variadic_split.cpp:121:
While validating node 'v1::VariadicSplit VariadicSplit_668 (StatefulPartitionedCall/CenterNet_MobilenetV3large/postprocess_layer/GatherV2[0]:f32{?,1,100,?}, 23315_const[0]:i64{}, 12213_const[0]:i64{2}) -> (dynamic?, dynamic?)' with friendly_name 'VariadicSplit_668':
Total length of splits: 2 must match the length of the chosen axis: 1

I was also able to localize the error, it is coming from the operation :

ys, xs = ys + reg[..., 1:2], xs + reg[..., 0:1]

here, xs and ys are tensors with shape (batch,100,1) and reg is a tensor with shape (batch,100,None).

By skipping this operation, ie.ReadNetwork() is able to load the graph

I have attached .xml, .bin and .mapping files.

Any help would be great.

Regards

Shubham Gupta

[Adli]

Hi Shubham Gupta,

Thank you for reaching out to us. We want to replicate this case on our device. Please the following information here:

• Ubuntu Version
• Python Version

Regards,

Adli

[Gupta__Shubham]

Hi Adli,

Ubuntu version is 18.04

python version is 3.6.9

I can also attach the tensorflow model if required.

Regards

Shubham Gupta

[Gupta__Shubham]

The error went away when i reshaped the "reg" tensor before the operation:

ys, xs = ys + reg[..., 1:2], xs + reg[..., 0:1]

by reg = tf.reshape(reg, (batch,-1,2))

but now new error is coming.

So in tensorflow  final operations looks like:

reg = tf.reshape(reg,(batch,-1,2))
wh = tf.reshape(wh,(batch,-1,2))

//////xs and ys are tensors of shape(batch,100,1)

ys, xs = ys + reg[..., 1:2], xs + reg[..., 0:1]

wh = tf.math.abs(wh)
ymin = ys - wh[..., 1:2] / 2
xmin = xs - wh[..., 0:1] / 2
ymax = ys + wh[..., 1:2] / 2
xmax = xs + wh[..., 0:1] / 2

return xmin,xmax,ymin,ymax

with this now I'm getting an error:

terminate called after throwing an instance of 'InferenceEngine::details::InferenceEngineException'
what(): StatefulPartitionedCall/CenterNet_MobilenetV3large/postprocess_layer/sub_1 has zero dimension which is not allowed

This error also goes away if I only return ymin or ymax. when i try to return both this error is coming.

I have attached the new openvino IR files for this error.

Regards

Shubham Gupta

[Gupta__Shubham]

Following the last message, when i only return xmin,xmax,ymin

......

wh = tf.math.abs(wh)
ymin = ys - wh[..., 1:2] / 2
xmin = xs - wh[..., 0:1] / 2
ymax = ys + wh[..., 1:2] / 2
xmax = xs + wh[..., 0:1] / 2

return xmin,xmax,ymin

everything works fine. it doesn't matter if you return ymin or ymax.

I have attached the openvino files for this also

Regards

Shubham Gupta

[Adli]

Hi Shubham Gupta,

Thank you for your prompt response. A quick check, did you refer to this documentation on how to implement custom layers for Tensorflow models? The article also provides an example of the "Hybrid-CS-Model-MRI" TF model with 3 unsupported operations.

Additionally, please also check this link for custom layer implementation in Model Optimizer. https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_customize_model_optimizer_Customize_Model_Optimizer.html

Regards,

Adli

[Gupta__Shubham]

Hi Adli,

All the operations I used are supported by openvino. I'm facing errors with basic operations(+,-,slicing etc.).

Also, model optimizer is able to convert the model to IR.

And following to my last message, where I mentioned that only when I try to return both ymin and ymax from the model output, then only I'm getting an error otherwise its working fine, this behaviour is very random.

Also, if i return tensor "reg" and "wh" from the model and calculate xmin,ymin,xmax,ymax as a post processing step, everything works fine including the final results, which clearly states that up to this point nothing is wrong. 

Calculation of xmin,ymin,xmax,ymax from "reg" and "wh" tensor only requires  two "+" and two "-" operations, that is where it start creating random behaviour.

Regards

Shubham Gupta

Regards

Shubham Gupta 

[Adli]

Hi Shubham Gupta,

We are currently investigating this issue and will get back to you as soon as possible.

Regards,

Adli

[Munesh_Intel]

Hi Shubham,

We found that this issue probably happens due to empty tensors. As such, please make sure that all the variables in your custom trained model do not return empty tensors. OpenVINO does not currently support empty tensors.  

Regards,

Munesh

[Gupta__Shubham]

Hi Munesh,

Tensors are not empty, if you look into my previous comments.

reg = tf.reshape(reg,(batch,-1,2))
wh = tf.reshape(wh,(batch,-1,2))

//////xs and ys are tensors of shape(batch,100,1)

ys, xs = ys + reg[..., 1:2], xs + reg[..., 0:1]

return xs,ys,wh

this works, i.e if I return xs,ys,wh tensor and calculate xmin, ymin, xmax, ymax  as a part of post processing step, everthing works fine.

If the problem was due to empty tensors then above should also not work.

The problem arises when i try to calculate xmin,ymin,xmax,ymax in graph itself, by

.....

wh = tf.math.abs(wh)
ymin = ys - wh[..., 1:2] / 2
xmin = xs - wh[..., 0:1] / 2
ymax = ys + wh[..., 1:2] / 2
xmax = xs + wh[..., 0:1] / 2

return xmin,xmax,ymin,ymax

I have also attached different IR files for replicating the issue.

Also, If i only calculate one of xmin,ymin,xmax,ymax then also everything works. Problem is only coming when i try to return all four values. If the problem was due to the empty tensor, returning only one value should also not work.

Regards

Shubham Gupta 

[Munesh_Intel]

Hi Shubham,

Can you provide your TensorFlow model so that we can take a look at it? Please also share more details about your custom model, the topology of your model, the version of TensorFlow that your model was trained in, and environment details (version of TensorFlow).

Regards,

Munesh

[Munesh_Intel]

Hi Shubham,

Thank you for your question. If you need any additional information from Intel, please submit a new question as this thread is no longer being monitored.

Regards,

Munesh