Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6447 Discussions

benchmark_app fails when running with "-infer_precision CPU:bf16"

ClinB
Beginner
1,048 Views

Hi all:

 

I tried to run OpenVINO benchmark_app on my AWS EC2 instance but it failed running with option "-infer_precision CPU:bf16"

 

AWS instance:

  • Type: m7i-flex large
  • OS: Windows Server 2019 (AMI ID: ami-065b889ab5c33720e )
  • CPU: Intel Xeon Platinum 8488C, 1 core, 2 threads (Code name: Sapphire Rapids)
  • Memory: 8G
  • HDD: 30G

(I thought this CPU should support precision FP16 & FP32)

 

Model:

  • asl-recognition-0004
  • downloaded by "omz_downloader --name asl-recognition-0004 --precisions FP16 --output_dir myLocalPath"

 

What I did:

  • (create and activate python 3.8 environment)
  • pip install openvino==2022.3
  • pip install openvino-dev==2022.3
  • omz_downloader --name asl-recognition-0004 --precisions FP16 --output_dir myLocalPath
  • benchmark_app -infer_precision CPU:bf16 -m myLocalPath\asl-recognition-0004.xml
  • (then it failed)

1.JPG

 

Although precision of the model is FP16, I tried to set -infer_precision to CPU:f32, then it ran successfully.

2.JPG

 

I also downloaded asl-recognition-0004 FP32 ver. and benchmark_app still fails when running with "-infer_precision CPU:bf16"

 

It now seems benchmark_app always failed when running with option "-infer_precision CPU:bf16", no matter which precision of input model.

 

Sorry but I failed to find related documents about this case.

 

Best Regards

Clin B

0 Kudos
3 Replies
Aznie_Intel
Moderator
1,007 Views

Hi Clin B,

 

I'm able to run both FP16 and FP32 infer_precision with Benchmark_App with the latest OpenVINO 2023.0 version. Please refer below:

1) benchmark_app -infer_precision CPU:bf16 -m \<Download_dir>\FP16\asl-recognition-0004.xml

FP16FP16

 

2)  benchmark_app -infer_precision CPU:f32 -m \<Download_dir>\FP32\asl-recognition-0004.xml

FP32FP32

 

 

Regards,

Aznie

 

0 Kudos
longfei98
Beginner
893 Views

I also have similar issue. It seems that different cpu type and windows kernel type have different behaviors. Sometimes it crashes, sometimes not. Strange issue.

0 Kudos
Aznie_Intel
Moderator
875 Views

Hi Clin B and longfei98,

 

This thread will no longer be monitored since we have provided a solution. If you need any additional information from Intel, please submit a new question.t

 

 

Regards,

Aznie


0 Kudos
Reply