Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.

benchmark_app fails at step 8

Brijender_B_Intel
511 Views

Not sure, what is missing:

 

(base) c:\AI\CodeGen\cg350model\ov_model>benchmark_app -m openvino_model.xml
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading OpenVINO Runtime
[ INFO ] OpenVINO:
[ INFO ] Build ................................. 2024.1.0-15008-f4afc983258-releases/2024/1
[ INFO ]
[ INFO ] Device info:
[ INFO ] CPU
[ INFO ] Build ................................. 2024.1.0-15008-f4afc983258-releases/2024/1
[ INFO ]
[ INFO ]
[Step 3/11] Setting device configuration
[ WARNING ] Performance hint was not explicitly specified in command line. Device(CPU) performance hint will be set to PerformanceMode.THROUGHPUT.
[Step 4/11] Reading model files
[ INFO ] Loading model files
[ INFO ] Read model took 173.06 ms
[ INFO ] Original model I/O parameters:
[ INFO ] Model inputs:
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [?,?]
[ INFO ] attention_mask (node: attention_mask) : i64 / [...] / [?,?]
[ INFO ] position_ids (node: position_ids) : i64 / [...] / [?,?]
[ INFO ] beam_idx (node: beam_idx) : i32 / [...] / [?]
[ INFO ] Model outputs:
[ INFO ] logits (node: __module.model/aten::to/Convert) : f32 / [...] / [?,?,51200]
[Step 5/11] Resizing model to match image sizes and given batch
[ INFO ] Model batch size: 1
[Step 6/11] Configuring input of the model
[ INFO ] Model inputs:
[ INFO ] input_ids (node: input_ids) : i64 / [...] / [?,?]
[ INFO ] attention_mask (node: attention_mask) : i64 / [...] / [?,?]
[ INFO ] position_ids (node: position_ids) : i64 / [...] / [?,?]
[ INFO ] beam_idx (node: beam_idx) : i32 / [...] / [?]
[ INFO ] Model outputs:
[ INFO ] logits (node: __module.model/aten::to/Convert) : f32 / [...] / [?,?,51200]
[Step 7/11] Loading the model to the device
[ INFO ] Compile model took 12218.41 ms
[Step 8/11] Querying optimal runtime parameters
[ INFO ] Model:
[ ERROR ] Exception from src\inference\src\cpp\compiled_model.cpp:145:
Exception from src\plugins\intel_cpu\src\node.cpp:503:
Node Broadcast_141508 contains less child edges than 1

Traceback (most recent call last):
File "C:\Users\bbharti\AppData\Local\anaconda3\Lib\site-packages\openvino\tools\benchmark\main.py", line 452, in main
value = compiled_model.get_property(k)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Exception from src\inference\src\cpp\compiled_model.cpp:145:
Exception from src\plugins\intel_cpu\src\node.cpp:503:
Node Broadcast_141508 contains less child edges than 1

0 Kudos
5 Replies
Vipin_Singh1
Moderator
469 Views

Hi Brijender, we would like to inform you that we are routing your query to the dedicated team for further assistance.


0 Kudos
Megat_Intel
Moderator
430 Views

Hi Brijender_B_Intel,

Thank you for reaching out to us.

 

We believe the error you received is due to the model itself. Could you please provide us with the model and the model information for further investigation?

 

On my end, I was able to run the OpenVINO™ 2024.1.0 benchmark_app on the model efficientdet-d1-tf:

 eddicientdet-d1-tf.png

 

 

Regards,

Megat

 

 

0 Kudos
Brijender_B_Intel
418 Views

hi.

thanks for looking into this. The model is  hugging face model  salesforce/codegen-350M-muli converted to openvino through optimum command:

optimum-cli export openvino --model salesforce/codegen-350M-multi openvino_model


thanks
Brijender 

 

0 Kudos
Megat_Intel
Moderator
385 Views

Hi Brijender,

Thank you for providing us with the details.

 

For your information, I encountered a similar error when running the benchmark_app on the CPU device:

cpu error .png

 

However, I successfully run the benchmark_app on the GPU device. You will need to specify the data shape since the model is dynamic. Please try to run the benchmark_app on the GPU device and specify the shape value using the parameter -shape "input_ids[1,1],attention_mask[1,1],position_ids[1,1],beam_idx[1]":

 gpu-success.png

 

If you encountered the error "Incompatible MatMul matrix dimension", please reduce the number of iterations. On my end, selecting -niter 500 resolves the error.

 

On another note, I was able to load the model and successfully ran the inference using the code below for both CPU and GPU devices. The error seems to be related to the benchmark_app. We will investigate this issue and get back to you soon.

 llm success normal .png

from optimum.intel import OVModelForCausalLM
from transformers import AutoTokenizer

model = OVModelForCausalLM.from_pretrained("openvino_model")
#model.to("GPU") - for GPU device
prompt = "def hello_world():"
tokenizer = AutoTokenizer.from_pretrained("openvino_model")
input_ids = tokenizer(prompt, return_tensors="pt").input_ids
generated_ids = model.generate(input_ids, max_length=128)
print(tokenizer.decode(generated_ids[0], skip_special_tokens=True))

 

 

Regards,

Megat

 

0 Kudos
Brijender_B_Intel
371 Views

hi Megat.

Thanks for debugging it. Yes model inference runs fine but this tool was giving error. Looking forward for cpu fix.

-regards

Brijender

0 Kudos
Reply