Myriad X vs Myriad 2 benchmarks on OpenVINO examples

[Wiegersma__Aalzen]

I've tested some of the OpenVINO examples provided by Intel on both the Myriad X(NCS2) and Myriad 2(NCS). Performance of the Myriad X is, with the exception of classification and pose estimation, somewhat underwhelming compared to the Myriad 2. In some cases(face detection, semantic segmentation) the Myriad 2 is even significantly faster than the Myriad X.

• What could be the reason for the Myriad X's performance?
• Are additional optimizations for Myriad X possible?
• Is there a possibility Myriad X performance will increase significantly in future versions of OpenVINO?

Benchmarks(OpenVINO R4, Ubuntu 16.04, i5 4670 processor, USB 3 port) are listen below:

Classification

./classification_sample -i /opt/intel/computer_vision_sdk/deployment_tools/demo/../demo/car.png -m /home/user/openvino_models/ir/squeezenet1.1/FP16/squeezenet1.1.xml -d MYRIAD

Inference time:
- Myriad X            9.5600542 ms
- Myriad 2            29.8256800 ms
- CPU i5 4670 (FP32)  2.8685580 ms

Object detection and classification

./security_barrier_camera_demo -d MYRIAD -d_va MYRIAD -d_lpr MYRIAD -i /opt/intel/computer_vision_sdk/deployment_tools/demo/../demo/car_1.bmp -m /opt/intel/computer_vision_sdk/deployment_tools/demo/../intel_models/vehicle-license-plate-detection-barrier-0106/FP16/vehicle-license-plate-detection-barrier-0106.xml -m_va /opt/intel/computer_vision_sdk/deployment_tools/demo/../intel_models/vehicle-attributes-recognition-barrier-0039/FP16/vehicle-attributes-recognition-barrier-0039.xml -m_lpr /opt/intel/computer_vision_sdk/deployment_tools/demo/../intel_models/license-plate-recognition-barrier-0001/FP16/license-plate-recognition-barrier-0001.xml

Inference time:
- Myriad X            98.345 ms
- Myriad 2            106.327 ms
- CPU i5 4670 (FP32)  55.163 ms

Single shot detector

./object_detection_sample_ssd -m /opt/intel/computer_vision_sdk/deployment_tools/intel_models/person-detection-retail-0013/FP16/person-detection-retail-0013.xml -i ~/Videos/MVI_0065_1_0083.jpg -d MYRIAD

Inference time:
- Myriad X            159.313 ms
- Myriad 2            170.651 ms
- CPU i5 4670 (FP32)  15.3816 ms

Face detection

./interactive_face_detection_demo -m /opt/intel/computer_vision_sdk/deployment_tools/intel_models/face-detection-adas-0001/FP16/face-detection-adas-0001.xml -i ~/Videos/MVI_0065_1.avi -d MYRIAD

Inference time:
- Myriad X            155 ms
- Myriad 2            124 ms
- CPU i5 4670 (FP32)  15 ms

Semantic segmentation

./segmentation_demo -m /opt/intel/computer_vision_sdk/deployment_tools/intel_models/road-segmentation-adas-0001/FP16/road-segmentation-adas-0001.xml -i ~/Videos/MVI_0065_1_0083.jpg -d MYRIAD

Inference time:
- Myriad X            726.45 ms
- Myriad 2            438.159 ms
- CPU i5 4670 (FP32)  54.7694 ms

Pose estimation

./human_pose_estimation_demo -m /opt/intel/computer_vision_sdk/deployment_tools/intel_models/human-pose-estimation-0001/FP16/human-pose-estimation-0001.xml -i ~/Videos/train.avi -d MYRIAD

Inference time:
- Myriad X            188 ms
- Myriad 2            365 ms
- CPU i5 4670 (FP32)  45 ms

[Dal_Ben__Mattia]

I got the same results. Is there any update on this?

[Wiegersma__Aalzen]

I've ran the examples again on OpenVino R5.01, Ubuntu 18.04, Core i5 4670 CPU:

classification_sample

Myriad X FP16  9.5 ms
Myriad 2 FP16 32.8 ms
CPU        FP32   2.9 ms

security_barrier_camera_demo

Myriad X FP16 53.8 ms
Myriad 2 FP16 80.0 ms
CPU        FP32 17.1 ms

object_detection_sample_ssd

Myriad X FP16 156.4 ms
Myriad 2 FP16 182.8 ms
CPU         FP32  13.5 ms

interactive_face_detection_demo

Myriad X FP16 133.1 ms
Myriad 2 FP16 146.0  ms
CPU         FP32  15.0 ms

segmentation_demo

Myriad X FP16 492.2 ms
Myriad 2 FP16 481.6 ms
CPU         FP32  53.0 ms

human_pose_estimation_demo

Myriad X FP16 192.0 ms
Myriad 2 FP16 384.6 ms
CPU         FP32  44.8 ms

There is some improvement, Myriad X is as fast or faster than Myriad 2. On a custom trained dataset the performance of a single shot detector on Myriad X is about 16 fps in asynchronous mode and the classification performance is pretty good. CPU performance is very good too.

[Zhenzhen_R_Intel]

@Wiegersma, Aalzen,

Where can I get the OpenVINO R5.0.1 binary package?

Thanks.

[nikos1]

@ Ren, Zhenzhen

 > Where can I get the OpenVINO R5.0.1 binary package?

Please try https://software.intel.com/en-us/openvino-toolkit/choose-download

[whatthisismyname]

I didnt test the demos but my script from NCSv1 to NCSv1 had a very good improvement.
I can also notice a way better performance on the pi than TFlite.

[rachel_r_Intel]

@nikos

Thank you!

[Kelly__Marcus]

So the i5 CPU is faster that both the AI chips in all the examples?