NCS2 Async request doesn't return result

[Sharma__Rajat]

We have 2 NCS2 connected to a RPi board. We have created 2 networks and are running Async request on both of the networks. After around 5 minutes the NCS2 stops giving result for the async request and the code blocks waiting for the result infinitely. We are using python API with wait(FOREVER) after starting 4 async requests. We have following query

a) How can we ensure that both models runs on different NCS2 devices?

b) Is there a known problem in running multiple models in Async mode on NCS2 devices?

c) Can we run 2 models in Async mode on single NCS2 device?

[Shubha_R_Intel]

Dear Sharma, Rajat,

My first thought is that maybe you're bumping into a memory limitation on your NCS2 device. Are one or both of your models huge ?  Let me answer your questions.

a) How can we ensure that both models runs on different NCS2 devices? Well, the VPU Plugin handles this for you. You don't need to assign tasks or workloads to a certain device. The plugin through cooperation with Inference Engine knows how to balance the tasks. If both NCS2 devices are running though, they will definitely both be hotter to the touch (just touch their surface)

b) Is there a known problem in running multiple models in Async mode on NCS2 devices?

No, there is no problem. This should work

c) Can we run 2 models in Async mode on single NCS2 device?

You certainly can do this. As aforementioned, the common limitation is model size if it doesn't work.

Hope it helps,

Thanks,

Shubha