I did some performance comparison using OpenVino for CPU (i5-8350U) vs GPU (Intel UHD Graphics 620)
I am running a simple inference (1x3x224x224 image using ResNet18). What surprises me is that I do not see any performance difference between CPU and GPU. Both are giving average inference time of around 15ms.
Could I be missing something?
Dear Jian S.,
This is plausible. You are likely not missing anything. We have an OpenVino sample which exactly is meant to perform benchmark experiments, it's called the benchmark app . Once cannot generalize based upon a single test but the benchmark app gives you easy options to try different scenarios. You are using an I5 CPU but I can tell you that in my experience, I7 CPU actually beats most GPUs for OpenVino inference.