Hi everybody, i have use openVINO for inference detect object. When i run 1 thread or 1 process, time for detect object in a image ~ 0.01s, but when use multi thread or multi process or add any process to cpu, time for detect image increase, i don't understand. Because when run python i use command task -c with core diffenrence, but speed decrease. thank you!
Multi threading improves performance by allowing multiple CPUs to work on a problem at the same time; but it only helps if two things are true: as long as the CPU speed is the limiting factor (as opposed to memory, disk, or network bandwidth) and so long as multithreading doesn't introduce so much additional work
Having more threads than what your CPU supports you are actually serializing and not parallelizing. The more threads you have the slower your system will be.
You can try to redesign your algorithm to be supporting Parallel computing and execute parts of your
algorithm simultaneously on multiple CPUs (using multiple threads).
You can refer here if you want to find out more on Openvino model optimization:
Hope my answer helps!