Hello, I am studying the Media SDK using the provided sample_decode application. One thing that strikes me as odd is the use of the MFXVideoDECODE::DecodeFrameAsync() and MFXVideoSession::SyncOperation(). Indeed, if I understand well, the code boils down to this (in CDecodingPipeline::RunDecoding()):
1. Start async decode operation
2. If it doesn't return any error, wait for it to finish
3. Goto 1 until all data is decoded
It seems counter-productive to start an asynchronous operation only to block immediately after. It also doesn't seem possible to register a callback to know when the operation completes exactly. Basically this looks like very inefficient code. I have experience with the NVCUVID decoder and the typical setup is to have two threads, one that feeds encoded frames and one that consumes decoded frames; the producer thread is always waiting on the decoder to signal that a surface is available for decoding, so the decoder can always be working at full potential. Is it possible to work with the Intel Media SDK decoder in such a way?
Intel Media SDK provides a completely asynchronous API. Users are not forced to synchronize after each call, like for instance DecodeFrameAsync, instead any number of concurrent asynchronous calls can be submitted before sync, assuming enough surface buffers have been allocated.
To better understand the sync v.s async usages of the SDK I strong encourage you to take a look at the Media SDK tutorial (http://software.intel.com/en-us/articles/intel-media-sdk-tutorial), which starts out with simple synchronous examples and gradually builds on these to achieve desired level of asynchronicity.
There are many architectural ways of implementing asynchronous software. One approach is using call-backs as you mention. That is a fine approach, but the Media SDK decided to utilize alternate asynchronicity method.
Specifically, regarding Media SDK Decode. We do not provide any samples on how to implement full async decode similar to the async encode sample. But developers are free to extend on the code to implement such use case if needed. The other samples can be used as a blueprint. However, one thing to keep in mind is that the decode sample still has good performance due to the nature of the DecodeFrameAsync call, which does schedule decode of many frames in a single call to DecodeFrameAsync, if the input bitstream contains several compressed frames.
Thank you for the insightful reply, I checked the simple_decode_d3d_async example which shows how to start several async operations at a time, but that's not really what I'm looking for. I'm trying to see how to integrate this new decoder in an existing application where there's already a layer of abstraction over several decoders, and we use a callback asynchronous model where it's the decoder's responsibility to tell when it's done decoding a frame. To integrate the Intel Media SDK decoder in this model, it seems I'd need to have the implementation spawn a thread just to call Sync() all the time and call the callback when it returns something. Is this correct or is there a more idiomatic way to get callbacks from the decoder?
Unfortunately there is no mechanism part of the SDK that implements the callback approach. To integrate with your framework you will have to wrap the SDK API with you own preferred callback layer.