Re:Is it possible that certain Tensorflow functions cannot be used?

KojiroMachi · ‎03-01-2021

I am currently using OpenVINO's Optimizer using tensorflow. The following error occurs when performing model transformation.
My prediction is that some modules with tensorflow may not be available.

<ERROR>
=======================================================

[ WARNING ] Please set version attribute for node StatefulPartitionedCall/vae_mlp/encoder/z/random_normal/RandomStandardNormal with type=UNKNOWN

[ ERROR ] Elementwise operation StatefulPartitionedCall/vae_mlp/encoder/z/mul_1 has inputs of different data types: <class 'numpy.float32'> and <class 'numpy.int32'>

[ ERROR ] List of operations that cannot be converted to Inference Engine IR:

[ ERROR ] RandomStandardNormal (1)

[ ERROR ] StatefulPartitionedCall/vae_mlp/encoder/z/random_normal/RandomStandardNormal

[ ERROR ] Part of the nodes was not converted to IR. Stopped. For more information please refer to Model Optimizer FAQ, question #24. (https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html?question=24#question-24)
=======================================================

The backend.random_normal part could not be converted in the content of the error. Please tell us whether or not it can be converted, and if so, how to deal with it.

Gopika_Intel · ‎03-01-2021

Hi.

Thank you for reaching out to us.

As checking your issue, we could see that your issue is related to OpenVINO toolkit. We have a dedicated forum to handle those specific issues. So, we are moving the thread to OpenVINO forum.

Link : https://community.intel.com/t5/Intel-Distribution-of-OpenVINO/bd-p/distribution-openvino-toolkit

Regards

Gopika

Munesh_Intel · ‎03-02-2021

Hi Kojiro,

Thanks for reaching out to us. Yes, you’re right. “RandomStandardNormal” is not a supported TensorFlow operation in OpenVINO Model Optimizer.

TensorFlow Supported Operations are listed here:

https://docs.openvinotoolkit.org/2021.2/openvino_docs_MO_DG_prepare_model_Supported_Frameworks_Layers.html#tensorflow_supported_operations

For unsupported operations, you can create your own implementation for existing or completely new operation. Please refer to the following page, which illustrates the workflow for running inference on topologies featuring custom operations:

https://docs.openvinotoolkit.org/2021.2/openvino_docs_HOWTO_Custom_Layers_Guide.html

Please share more information about your model, is it an object detection/classification model, if custom model what type of layers does the model use, command given to Model Optimizer to convert the trained model to Intermediate Representation (IR), and also environment details (versions of OS, TensorFlow, Python, CMake, etc.).

Regards,

Munesh

KojiroMachi · ‎03-11-2021

Thank you for your reply.
I understand that Tensorflow's Random_normal is not supported.
Currently, we are inserting the following processing into the AI model.

==============================================
z = Lambda(
　lambda x: x[0] + K.exp(0.5*x[1]) + K.random_normal(shape=(K.shape(x[0])[0], K.int_shape(x[0])[1])),
　output_shape=(latent_dim,)
)([z_mean, z_log_var])
==============================================

The function that cannot be model-transformed is K.random_normal.
What should I do if I create it myself?
I would appreciate it if you could tell me how to change the above function.

Kojiro Machi

Munesh_Intel · ‎03-15-2021

Hi Kojiro,

There are basically three steps to support inference of a model with custom operation, as mentioned in the following link:

https://docs.openvinotoolkit.org/2021.2/openvino_docs_HOWTO_Custom_Layers_Guide.html#custom_operation_support_overview

Let's consider custom operations in Model Optimizer first. Firstly, you need to create extensions to support this unsupported operation.

Two types of the Model Optimizer extensions should be implemented to support custom operation at minimum, as mentioned here:

https://docs.openvinotoolkit.org/2021.2/openvino_docs_MO_DG_prepare_model_customize_model_optimizer_Customize_Model_Optimizer.html#extensions

i. Operation class for a new operation.

This class stores information about the operation, its attributes, shape inference function, attributes to be saved to an IR and some other internally used attributes. Refer to the “Model Optimizer Operation” section below for the detailed instruction on how to implement it.

https://docs.openvinotoolkit.org/2021.2/openvino_docs_MO_DG_prepare_model_customize_model_optimizer_Customize_Model_Optimizer.html#extension-operation

ii. Operation attributes extractor.

The extractor is responsible for parsing framework-specific representation of the operation and uses the corresponding operation class to update graph node attributes with necessary attributes of the operation. Refer to the "Operation Extractor" section below for the detailed instruction on how to implement it.

https://docs.openvinotoolkit.org/2021.2/openvino_docs_MO_DG_prepare_model_customize_model_optimizer_Customize_Model_Optimizer.html#extension-extractor

For NGraph operation, please refer to the following link for detailed instructions on how to implement it.

https://docs.openvinotoolkit.org/2021.2/openvino_docs_IE_DG_Extensibility_DG_AddingNGraphOps.html

Please refer to the following example chapter for step-by-step instruction on example implementation of using a custom operation on CPU.

https://docs.openvinotoolkit.org/2021.2/openvino_docs_HOWTO_Custom_Layers_Guide.html#enabling_magnetic_resonance_image_reconstruction_model

The sample implementation of the unsupported "ComplexAbs” operation in the example above is given here:

import numpy as np

 

from extensions.ops.elementwise import Pow

from extensions.ops.ReduceOps import ReduceSum

from mo.front.common.replacement import FrontReplacementOp

from mo.graph.graph import Graph, Node

from mo.ops.const import Const

 

 

class ComplexAbs(FrontReplacementOp):

   op = "ComplexAbs"

   enabled = True

 

   def replace_op(self, graph: Graph, node: Node):

       pow_2 = Const(graph, {'value': np.float32(2.0)}).create_node()

       reduce_axis = Const(graph, {'value': np.int32(-1)}).create_node()

       pow_0_5 = Const(graph, {'value': np.float32(0.5)}).create_node()

 

       sq = Pow(graph, dict(name=node.in_node(0).name + '/sq', power=2.0)).create_node([node.in_node(0), pow_2])

       sum = ReduceSum(graph, dict(name=sq.name + '/sum')).create_node([sq, reduce_axis])

       sqrt = Pow(graph, dict(name=sum.name + '/sqrt', power=0.5)).create_node([sum, pow_0_5])

       return [sqrt.id]

Regards,

Munesh

Munesh_Intel · ‎03-24-2021

Hi Kojiro,

This thread will no longer be monitored since we have provided references and example. If you need any additional information from Intel, please submit a new question.

Regards,

Munesh

KojiroMachi · ‎05-11-2021

Hi. Munesh

Is it possible to see the basic code of the extension code of RandomNormalStandard?

We apologize for the inconvenience, but if possible, we would like to ask.

Regards,

Kojiro