Batch with constant layer Openvino

[VictorMrt]

Hello,

I'm using OpenVINO to do inference on a intel CPU. I want to infer on a lstm model where there is 1 input layer and 2 constant layer.

model optimizer traduction: 

<layer id="0" name="input" type="Parameter" version="opset1">
<data shape="150,1,6" element_type="f32"/>
<output>
<port id="0" precision="FP32">
<dim>150</dim>
<dim>1</dim>
<dim>6</dim>
</port>
</output>
</layer>
<layer id="1" name="281_const" type="Const" version="opset1">
<data offset="0" size="512" shape="1,128" element_type="f32"/>
<output>
<port id="1" precision="FP32">
<dim>1</dim>
<dim>128</dim>
</port>
</output>
</layer>
<layer id="2" name="285_const" type="Const" version="opset1">
<data offset="0" size="512" shape="1,128" element_type="f32"/>
<output>
<port id="1" precision="FP32">
<dim>1</dim>
<dim>128</dim>
</port>
</output>
</layer>

(the dimension of input layer is [150 batch_size 6])

These 3 layers go in a LSTM layer for computation.

The problem is that if I want to change the batch size of this model for an inference so change [150 1 6] to [150 2 6] this error happen 

'v0::TensorIterator TensorIterator_862(Parameter_1861[0]:f32{150,2,6}, Constant_1[0]:f32{1,128}, Constant_2[0]:f32{1,128}) -> (f32{150,1,128}, f32{1,128}, f32{1,128})'

I cannot access to the constant layer to change the batch size of them with the API because the constant layer are not input layer, so not present in network.getInputsInfo().

I tried to include them in --input option in the model optimizer but an error happen :

[ ERROR ] Please contact Model Optimizer developers and forward the following information:
[ ERROR ] Exception occurred during running replacer "REPLACEMENT_ID (<class 'extensions.front.input_cut.InputCut'>)": 0
[ ERROR ] Traceback (most recent call last):
File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/utils/class_registration.py", line 286, in apply_transform
replacer.find_and_replace_pattern(graph)
File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/extensions/front/input_cut.py", line 34, in find_and_replace_pattern
add_input_ops(graph, graph.graph['user_shapes'], True)
File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/front/extractor.py", line 1307, in add_input_ops
add_input_ops_helper_before_infer_input_port(graph, smart_node, port, node_id, shape, data_type, inputs,
File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/front/extractor.py", line 1190, in add_input_ops_helper_before_infer_input_port
edges_to_remove.append((smart_node.in_node(port).id, smart_node.id))
File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/graph/graph.py", line 207, in in_node
return self.in_nodes(control_flow=control_flow)[key]
KeyError: 0

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/main.py", line 309, in main
ret_code = driver(argv)
File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/main.py", line 270, in driver
ret_res = emit_ir(prepare_ir(argv), argv)
File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/main.py", line 235, in prepare_ir
graph = unified_pipeline(argv)
File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/pipeline/unified.py", line 25, in unified_pipeline
class_registration.apply_replacements(graph, [
File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/utils/class_registration.py", line 334, in apply_replacements
apply_replacements_list(graph, replacers_order)
File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/utils/class_registration.py", line 320, in apply_replacements_list
apply_transform(
File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/utils/logger.py", line 124, in wrapper
function(*args, **kwargs)
File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/utils/class_registration.py", line 308, in apply_transform
raise Exception('Exception occurred during running replacer "{} ({})": {}'.format(
Exception: Exception occurred during running replacer "REPLACEMENT_ID (<class 'extensions.front.input_cut.InputCut'>)": 0

[ ERROR ] ---------------- END OF BUG REPORT --------------
[ ERROR ] -------------------------------------------------

Do you have any solution for me ? In the best case by not changing the model 

[VictorMrt]

I'm using OpenVINO 2020.4

[Soumya_M_Intel]

Hi VictorMrt,

Thanks for posting your concern in the forum. Please note, this forum topic is to provide support on post-installation queries of Intel® oneAPI Base Toolkit. However, be rest assured, as we are routing your post to the correct forum. Our internal team would get back to you there in a timely manner. Have a nice day ahead.

Regards,

Soumya

[Zulkifli_Intel]

Hello Victor Martin,

Greetings to you.

Please share with us your model and step to reproduce. Are you converting the model using model optimizer (MO)? if so please provide the command.

Regards,

Zulkifli

[Zulkifli_Intel]

Hello Victor Martin.

Thank you for your question. If you need any additional information from Intel, please submit a new question as this thread is no longer being monitored.

Regards,

Zulkifli