Problems in decoding live stream form ip camerra with low latency

[k_d_]

I'm trying to integrate Intel Media SDK decoder into an CCTV application wchich receives h.264 encoded frames from ip camera and expects raw frames as an output. Output frames are next presented on the screen, so decoding needs to be done with low latency. Input stream does not contain B frames.

I have met several problems:
- with AsyncDepth=1 QueryIoSurf sets the Requests.NumFramesSuggested to 7, wchich causes 7 frames latency in receiving decoded output frame (in CCTV application this is not acceptable); 
- when AsyncDepth is set to 1 many calls of DecodeFrameAsync (about 15-20%) returns staus MFX_WRN_DEVICE_BUSY (despite I am only using software implementation) and the output stream has very bad quality with lots of artifacts (especially when here is movement on camera). Setting AsyncDepth to 0 partialy resolves the problem but Requests.NumFramesSuggested is now 10.

Is there any possibility to reduce latency and increase quality of input stream?

The application previously was using IPP for decoding and didn't meet such problems.

[Petter_L_Intel]

Hi,

Some things to keep in mind, for low latency, beyond the requirement to configure the decoder for low latency. The encoded stream you receive must have no B-frames (as you stated), and the stream must also have DPB set to 1 to ensure decoder does not buffer frames (if you used Media SDK encode this would be controlled by mfxExtCodingOption::MaxDecFrameBuffering = 1).

Requests.NumFramesSuggested (in your case = 7) does not determine the decoder latency, it's reflecting the required # of internal decoder buffers, due to Media SDK and driver internal logic (# of buffers for decode may seem artificially high, but at this point they are required).

MFX_WRN_DEVICE_BUSY is a normal return code indicating that the underlying decoder is busy processing the frame. This use case is illustrated in the Media SDK samples.

If you are experiencing lack of low latency (essentially, the decoder not being able to deliver a frame directly after single successful DecodeFrameAsync + SyncOperation) then my guess is that there are issues with the received stream such as DPB not set to 1.

Also, assuming you receive a single h.264 frame at a time, note that you should set the decoded bitstream flag to MFX_BITSTREAM_COMPLETE_FRAME (in mfxBitStream::DataFlag) before calling DecodeFrameAsync().

Regards,
Petter