Big difference in performance using OpenVINO model on CPU vs Intel Compute Stick 2

[Letty]

I own an Intel Compute Stick 2 that I intend to use to process object detection networks.

After installing OpenVINO on my machine (Ubuntu 18.04), I tried running the object detection python demo on a video using one of the models available on the model zoo. When running it on the Intel stick, I would get a speed of around 7.5 fps, while running it on my laptop Intel CPU is a lot faster at 44 fps.

Even if my laptop is a decent gaming laptop, I was surprised by the fact that processing on the Intel stick is so much slower. I plan to use the Intel stick on another device, not my laptop, but I would like to understand why there is this big difference in performance.

[Peh_Intel · Accepted solution]

Hi Letty,

Intel® Neural Compute Stick 2 (NCS 2) is a USB stick that offers you access to neural network functionality, without the need for large, expensive hardware. It is a plug-and-play device, so you are ready to start prototyping right away.

However, talking about the performance of NCS 2 compared to the well-known CPUs or GPUs in the meaning of TFLOPS, it is still a hundred times lower, so don’t rely on it as an attachment to your laptop to replace the CPU plugin.

On a separate note, in order to improve some additional FPS, try launching NCS 2 on a device with USB 3.0 and running the demo with FP16 of the model. If the FPS didn’t have any increase, then it is already the optimum performance and this behaviour is expected.

Regards,

Peh

[Letty]

Thank you very much for your reply.

I don't plan to use it to replace my laptop CPU but to help processing on another device. I am running FP16 and USB 3.0 already so I guess this is already the optimum performance.

I will see if I can increase the performance in other ways, thank you.

[Peh_Intel]

Hi Letty,

This thread will no longer be monitored since we have provided an answer. If you need any additional information from Intel, please submit a new question. 

Regards,

Peh