Intel® Distribution of OpenVINO™ Toolkit
Community support and discussions about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all things computer vision-related on Intel® platforms.
5848 Discussions

running speed: MYRIAD slower than CPU; FP16 is comparable with FP32?

fariver
Beginner
488 Views

Hi

I fall into two confusion: 1) my network run slower on MYRIAD than CPU; 2).  the FP16 network runs comparable speed with its FP32 version on MYRIAD device without any speedup.

  FP16 FP32
MYRIAD 1.0966s 1.0958s
CPU 0.7722s 0.7883s

All the time consuming result is averaged by 10 iterations. Attachment is my new network definition. Is this the expected performance? Hoping for your help. 

Bellow is some detail imformation.

 

My environment info:

Neural Compute Stick2

MAC OS 10.13

openvino_2021.4.582

 

My test program:

from openvino.inference_engine import IECore

ie = IECore()

xml_path = './baseline.xml'

net = ie.read_network(model=xml_path)

exec_net = ie.load_network(network=net, device_name=MYRIAD) 

 

 

0 Kudos
4 Replies
Peh_Intel
Moderator
459 Views

Hi fariver,

 

Regarding the performance between CPU and Intel® Neural Compute Stick 2 (NCS2), you can have a look at this Benchmark Results. By looking into all the chosen models, the performance of CPU is always better than NCS2. Referring to these Supported Model Formats, the VPU plugins only support FP16 models.

 

 

Regards,

Peh


fariver
Beginner
440 Views

By looking into all the chosen models, the performance of CPU is always better than NCS2.

 


Hi Peh, 

As shown in Benchmark Results,  the NCS2 has much better performance than i5 CPU. However, the conclusion disappearance in my customized network. Can you help me profile what's happen? or it can also be helpful if you can tell me which operations run faster on CPU than MYRIAD. Hoping for replying.

fariver_0-1626503573319.png

 

 

Peh_Intel
Moderator
401 Views

Hi fariver,

 

Please note that the values in these two graphs are different. You can point to the bar to observe the exact value. Besides, also note that there are two comparison graphs in term of throughput (FPS) and value (FPS/$) as I noticed that you’re observing the value comparison graph.


Intel® Neural Compute Stick 2 (NCS 2) is a USB stick that offers you access to neural network functionality, without the need for large, expensive hardware. It is a plug-and-play device, so you are ready to start prototyping right away. However, the performance of CPU is always better than NCS2.

 

 

Regards,

Peh


Peh_Intel
Moderator
360 Views

Hi fariver,


This thread will no longer be monitored since we have provided answers. If you need any additional information from Intel, please submit a new question. 



Regards,

Peh


Reply