I fall into two confusion: 1) my network run slower on MYRIAD than CPU; 2). the FP16 network runs comparable speed with its FP32 version on MYRIAD device without any speedup.
All the time consuming result is averaged by 10 iterations. Attachment is my new network definition. Is this the expected performance? Hoping for your help.
Bellow is some detail imformation.
My environment info:
Neural Compute Stick2
MAC OS 10.13
My test program:
from openvino.inference_engine import IECore
ie = IECore()
xml_path = './baseline.xml'
net = ie.read_network(model=xml_path)
exec_net = ie.load_network(network=net, device_name=MYRIAD)
Regarding the performance between CPU and Intel® Neural Compute Stick 2 (NCS2), you can have a look at this Benchmark Results. By looking into all the chosen models, the performance of CPU is always better than NCS2. Referring to these Supported Model Formats, the VPU plugins only support FP16 models.
By looking into all the chosen models, the performance of CPU is always better than NCS2.
As shown in Benchmark Results, the NCS2 has much better performance than i5 CPU. However, the conclusion disappearance in my customized network. Can you help me profile what's happen? or it can also be helpful if you can tell me which operations run faster on CPU than MYRIAD. Hoping for replying.
Please note that the values in these two graphs are different. You can point to the bar to observe the exact value. Besides, also note that there are two comparison graphs in term of throughput (FPS) and value (FPS/$) as I noticed that you’re observing the value comparison graph.
Intel® Neural Compute Stick 2 (NCS 2) is a USB stick that offers you access to neural network functionality, without the need for large, expensive hardware. It is a plug-and-play device, so you are ready to start prototyping right away. However, the performance of CPU is always better than NCS2.