when executing multiple InferRequests simultaneously on a network on the GPU, the outputs get mixed up sometimes. There is some code reproducing the issue attached (bug.cpp)
What the Code does:
- Create a network with multiple InferRequests, where each InferRequest has the callback set to the same function
- Create 13 different inputs
- Start an inference whenever an InferRequest is available
- In the callback, compare the results with the result of the first inference for the image. If the results do not match, print to which inputs the results would match.
wrong i: 4, b:5, c:6, r:5
means, input is from image 4, output "b" is from image 5, output "c" is from image 6, output "r" is from image 5
This is the problem, we get outputs for another input and even the outputs are mixed up. The problem did not occur with a single InferRequest and also did not occur in CPU mode. It seems to be a race condition as mixed up outputs are rare and occur mostly when InferRequests are executed rapidly.
How to reproduce the test:
Run in docker with:
docker run -it --device /dev/dri:/dev/dri -u root -v /tmp/bug/:/bug --rm openvino/ubuntu20_data_dev:latest
Start test application with:
Is this problem known? Or is there something we overlooked?
I noticed that this issue is similar to this Github case:
We are investigating this and will further communicate through Github.
Hence, I'm closing this as a duplicate issue.