H264VideoDecoder::GetFrame Inconsistent

[Steve_Browne]

I would say this is probably a bug, but at the very least its a change from the old way it worked so it should be better documented.

Basically lets assume the decoder is only going to buffer one frame. What happens is you pass in the first frame to GetFrame and it returns UMC_ERR_NOT_ENOUGH_DATA as expected. The second time you pass in a frame it will return UMC_OK and your output buffer will be filled with the decompressed frame, however the input data will not have been read in. This is in contrast to how it worked in previous IPP sample code versions. To get around this you have to check the GetDataSize() of the input parameter and if it is non-zero then GetFrame didn't read it in and you'll have to call GetFrame again and assume its going to buffer the frame and not give you back another decoded frame. That should be a safe assumption, but this does go against what the documentation says about GetFrame: "An obtained decoded frame is supposed to be used only until the next GetFrame is called."

The other funky thing is that sometimes calling GetFrame with a NULL input will still return UMC_ERR_NOT_ENOUGH_DATA. If you call it a second time in a row with a NULL input then it will return UMC_OK and give you the decoded frame.

I can work around these, but I would think this should be handled internally in the decoder rather than the consumer having to worry about it.

[Pavel_V_Intel]

Good day.

What version of UMC do you use? For 7.0.x documentation says what "The decoder can use only part of the provided data and updates the data amount and the data pointer of UMC::MediaData instance", so you should check input buffer for emptiness.

That should be a safe assumption, but this does go against what the documentation says about GetFrame: "An obtained decoded frame is supposed to be used only until the next GetFrame is called."

It returned you a frame but didn't use input data, so you should use this frame until next call, I don't see contradiction here. And this document statement is true only if you don't use your own frame buffer.

The other funky thing is that sometimes calling GetFrame with a NULL input will still return UMC_ERR_NOT_ENOUGH_DATA. If you call it a second time in a row with a NULL input then it will return UMC_OK and give you the decoded frame.

This it is not suppose to do. Can you provide stream with such problem?

[Steve_Browne]

I'm upgrading from UMC 6.1.6 to 7.0.6. The comments I made are regarding UMC 7.0.6 and I'm comparing differences between 6.1.6.


First I'll talk about the NULL input issue before I ramble on about the other thing. You should be able to duplicate it with any H.264 stream, in fact, all you need is a single I frame. Basically any time you use more than 2 threads (3 or more) you should be able to duplicate it on the very first frame if you try to flush it out. It does not happen with just 1 or 2 threads. I don't see it very often with normal streaming and flushing, but if you were to just try to decode 1 I frame out of any given H.264 stream and flush it out immediately it behaves consistently like this:
int ret = m_pDecoder->GetFrame(&input, m_pOutput); // ret == UMC::UMC_ERR_NOT_ENOUGH_DATA
ret = m_pDecoder->GetFrame(NULL, m_pOutput);// ret == UMC::UMC_ERR_NOT_ENOUGH_DATA
ret = m_pDecoder->GetFrame(NULL, m_pOutput); // ret == UMC::UMC_OK


I agree that we should be checking the input buffer for emptiness and in fact we do. The complaint mainly stems from the fact that if a previously buffered from was decoded and returned in GetFrame then in 6.1.6 and lower the GetFrame call would still read in your input data. Now it will flush out the previous frame, but not read any data. This behavior is also slightly confusing to me as well since if the frame could be returned without reading any data then why was it not returned with the previous GetFrame call.


The only reason I complain is our usage, which I'm sure is probably fairly common involves popping the frame we're going to decode off the end of a queue and then sending it to the decoder. However, in this version there's no guaranteeing the decoder will use that data. Like I said I seem to be able to just call GetFrame with the same input data again and it always buffers it, but since this is not guaranteed and the docs say not to use the output again until you're done with it its a little worrying.


The main reason this is even coming up is because the change in behavior was throwing off our counter for tracking the number of frames the decoder has buffered. We use this counter so that we know which frame is actually being shown and for making our forward and reverse stepping more accurate. That's of course mainly our problem and I will surely fix it on our side, but I still thought the change in behavior was worth mentioning. It would actually be nicer if the decoder itself offered some way of determining how many frames it has buffered or maybe number of bytes buffered. Generally we only ever feed it one entire frame, but there was a camera we recently found that was packaging up one field as if it were a frame. While the decoder supports this our logic doesn't actually handle this properly and we have no way of knowing that's what happening internally.

[Pavel_V_Intel]

Similar problem was fixed in 7.0.7, you should try it.

This behavior is also slightly confusing to me as well since if the frame could be returned without reading any data then why was it not returned with the previous GetFrame call.

h264 decoder in umc 6.x and umc 7.x have different threading pipelines. The reason of this behaviour may be is that decoder doesn't have enough buffer to store new frame but between calls secondary thread has finished decoding of a buffered frame and it is ready for output.