Made a simple test comparison of NCS2, GPU and CPU on an i7-8750H laptop running Ubuntu-Mate 20.04 and OpenVINO R2021.1
Ubuntu-Mate 20.04, OpenVINO R2021.1 on i7-8750H.
Myraid NCS2 USB3 : ~14 fps (DNN_TARGET_MYRIAD)
OpenCV 4.5.0-openvino GPU : ~30 fps (DNN_TARGET_OPENCL_FP16)
OpenCV 4.5.0-openvino CPU : ~41 fps (DNN_TARGET_CPU)
So what CPU, or IOT class system, might have the best Bang/Buck to run the OpenCL GPU code?
You may use these data from this link to decide which hardware is best for your applications/solutions, or to plan your AI workload on the Intel computing already included in his solutions.
Hope this helps.
Intel will no longer monitor this thread since this issue has been resolved. If you need any additional information from Intel, please submit a new question.