Intel NCS (Neuron Computing Stick) Performance Issues

[UFO_Wu]

Hi Intel Experts,

Recently, I am using the OpenVINO 2018 R3 with NCS(Neuron Computing Stick) to make the object detection at Windows 10 platform.

I can successfully convert the tensor flow mobilenet (ssd_mobilenet_v1_coco_2017_11_17) to IR format with mo_tf.py tools. The object detection function works fine. But the performance of NCS is much lower than I expected. Here is data:

Model: ssd_mobilenet_v1_coco_2017_11_17 (mo_tf.py IR format)
Testing Tools: object_detection_demo_ssd_async.exe -i cam
CPU-FP32 ms (FPS): 36.46 ms (27.43 FPS)
GPU-FP32 ms (FPS): 58.88 ms (16.98 FPS)
NCS-FP16 ms (FPS): 874.40 ms (1.14 FPS)

The Test Environment:
CPU: Intel Core i5-7300U CPU@2.60GHz (RAM 8G)
GPU: Intel HD Graphics 620
OS: Windows 10 (x64) Pro 1703
OpenVINO: computer_vision_sdk_2018.3.343

Question:

1. Why the NCS performance is such lower than CPU and GPU
2. Is that specification of NCS or our model issues or TF --> IR conversion issues
3. Dose there any recommend object detection model for NCS
4. How to optimize the performance for the NCS

If there is document for the performance benchmark for NCS, please help to share with us.

The answer from Intel will be very helpful.

Thanks

[Qing__Albert]

It's as same as my test on NCS2. It looks like NCS is not very well performanced on CNNs calculation.