I've got a C++ OpenVino project run under Ubuntu 18.04.
It takes rtsp stream, decodes, makes some detection with default SSD architecture
Running just one instance of the project takes approx 30ms to process a frame.
2 instances - 80ms
3 - 150ms
4 - 210ms
5 - 280ms
Why it so non-linear running on a CPU only?
What is the proper way to run several instances on one CPU?
Have you looked at our async samples ? Please study our samples with _async in the name. Also please read the below blog-post (after downloading OpenVino 2019 R1). There are a few options you can try to improve OpenVino performance in alignment with physical CPU cores and thread instances.
Thanks for using OpenVino !