It seems like calling wait(-1) in the Python API is not blocking as it is supposed to be according to the documentation.
Also, it is unclear how to use it properly -
Assuming I have a writer thread and a reader thread and 2 requests, if the writer thread finished both requests,
the reader thread will see both as finished and will continue reading from them as if they were new outputs.
I did not find any mechanism to reset the InferenceRequest "finished" state.
For more complete information about compiler optimizations, see our Optimization Notice.