Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6392 Discussions

Optimize/Compile subclassed Keras model with multiple inputs #7565

indridi
Beginner
569 Views

This is continued as an issue on github, over here : https://github.com/openvinotoolkit/openvino/issues/7565 .

 

I have a quite unt​ypical use case. I am trying to run custom code in the guise of a Neural Network, optimizing and compiling using the OpenVino stack. The background is quite complicated, but basically, I am trying to utilize the power of MyriadX chips by disguising my code as a NN.

I am subclassing the Keras Model class, like this:

class MyIdentity(keras.Model):
    def __init__(self, name, shape: Tuple[int, int, int]):
        super().__init__(name=name)
        self.shape = tf.Variable(initial_value=shape, name="shape", trainable=False)
    def call(self, inputs):
        tmp = inputs['numbers'][0][0]
        # this is just a proof-of-concept code. I intend to do some operations combining a grid/matrix and some scalars in different ways
        return tf.cast(tmp*inputs['griddata'], tf.float16)

I export this to .pb using this code

shape = (1, 400, 640)
X = {'griddata': tf.ones(shape, dtype=tf.float16), 'numbers':tf.ones((1,3), dtype=tf.float16)}
model = MyIdentity(name="simplemodel", shape=shape)
full_model = tf.function(lambda x: model(x))
full_model = full_model.get_concrete_function(
    {'griddata': tf.TensorSpec(X['griddata'].shape, X['griddata'].dtype),
     'numbers': tf.TensorSpec(X['numbers'].shape, X['numbers'].dtype) } )
frozen_out_path = "../out_tf"
frozen_graph_filename = "tf_graph"
frozen_func = convert_variables_to_constants_v2(full_model)
frozen_func.graph.as_graph_def()
tf.io.write_graph(graph_or_graph_def=frozen_func.graph,
                  logdir=frozen_out_path,
                  name=f'{frozen_graph_filename}.pb',
                  as_text=False)

I optimize using the following call to the openvino tensorflow optimizer (after which I would call myriad_compile on the resulting graph.xml file)
python mo_tf.py --input_shape [1,400,640],[1,3] --input=griddata,numbers --input_model tf_graph.pb
The mo_tf.py fails with the following error.

[ ERROR ]  Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.front.user_data_repack.UserDataRepack'>): No node with name griddata

I have tried several different approaches to achieve this, the one above is the one that looks most sensible to me. If I use only one, unnamed input, I can manipulate it in the call() function and return in successfully, running a compiled blob on a myriadX chip. However, I have not been able to find a way get things to optimize or compile when using multiple inputs, despite trying several different approaches. The interwebs are unusually quiet about this, so Google has not been my friend on this issue.

Greetings,
Indriði

0 Kudos
2 Replies
Iffa_Intel
Moderator
537 Views

Greetings,


This case is the same as Github Case #7565 : https://github.com/openvinotoolkit/openvino/issues/7565


Please help to refer to that thread instead since we had provided the answer there.



Sincerely,

Iffa


0 Kudos
indridi
Beginner
514 Views

Thanks Iffa,

 

I wasn't sure which venue was the correct one. Strictly speaking, this isn't an issue with openvino, but more of a usage question. But I'll take it to github, if that's the preferred location for stuff like this.

 

Greetings,

Indriði

0 Kudos
Reply