Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Sapiain__Roberto
Beginner
126 Views

Question about thread/process safety of MYRIAD target (and maybe others)

Hi.

In a near future I'm going to see this problem, definitely.

I want to know if the drivers for NCS2 (and other targets, like OpenCL or OpenCL_FP16) have thread/process safety, when doing inference. If there is some sort of thread/process safe queue in the drivers that will just allow me to write data to perform inference (on one device, without changing ever the model loaded in the device after a 1st inference).

Or if I have to use some sort of semaphores/mutex (I'm coding in python currently, maybe later in C++) in my application.

This, thinking in the case I'll need to use multiprocessing or multi-threading to speed up frame processing, whe using Inference Engine backend.

Thank you in advance.

 

0 Kudos
2 Replies
Shubha_R_Intel
Employee
126 Views

Dear Sapiain, Roberto,

I hope the Integrate the Inference Engine Doc answers your questions. InferRequest, for example, is completely thread-safe. In general, Inference Engine calls you'd want to make are thread-safe. What does thread-safe mean ? To use the Core Inference Engine API from different threads, you don't need mutexes and semaphores and stuff.

Please see this excerpt from the doc:

Both requests are thread-safe: can be called from different threads without fearing corruption and failures.

Multiple requests for single ExecutableNetwork are executed sequentially one by one in FIFO order.

Hope it helps,

Shubha

Sapiain__Roberto
Beginner
126 Views

Hi Shubha.

Thank you very much for the reply.

king regards.

Reply