MFXVideoDECODE_DecodeFrameAsync return values

[Anton_Z_]

Hello.

I have strange (in my opinion) behaviour of MFXVideoDECODE_DecodeFrameAsync. On h264 stream sometimes it returns MFX_ERR_NONE (MFXVideoCORE_SyncOperation is ok too), but the internal mfxBitstream::DataOffset is zero. Moreover i have no valid data in the decoded surface.

I am not sure, can i rely on DataOffset field at all, anyway, just for test purpose i just retry call to MFXVideoDECODE_DecodeFrameAsync on the same mfxBitstream data several times until DataOffset will get the size of entire data (NALU) i feeded. Soon or later (2-8 retries) this happends and i have an output surface.

The question is why this can happends? Can i get more detailed debug information from the libs?

[Nina_K_Intel]

Hi Anton,

There's no debug info you can get from our libs, best source of information are samples and documentation (the spec - mediasdkman.pdf). To understand whether the output surface is ready after decoder the app needs to rely solely on return statuses from DecodeFrameAsync and SyncOperation. ERR_NONE after DecodeFrameAsync means you can call SyncOperation. ERR_NONE after SyncOperation means the data in surface_out is ready to use. Although I agree in your case it is logical to look at mfxBitstream.DataOffset for debug purposes, to understand what's happening. If both calls return ERR_NONE and there's no valid output - it would be a severe bug. But could you please double check that it is exactly ERR_NONE after both calls? Not a warning status (> ERR_NONE)? What you describe really looks similar to MFX_WRN_DEVICE_BUSY situation, when you just need to repeat the same call several times and eventually you get the decoded surface.

Regards,

Nina

[Anton_Z_]

Yes, i checked that several times - MFX_ERR_NONE is returned in both cases. FYI the code snippet:

	mfxSyncPoint syncp;

        while(true)
	{

...

		s = MFXVideoDECODE_DecodeFrameAsync(m_session, &m_bs, m_surfaces[surface_idx], &m_pout_surface, &syncp);

		switch(s)
		{
		case MFX_WRN_DEVICE_BUSY:
			this_thread::sleep_for(chrono::milliseconds(1));
		case MFX_ERR_MORE_SURFACE:
			continue;
		case MFX_WRN_VIDEO_PARAM_CHANGED:
		case MFX_ERR_MORE_DATA:
			if (sample.size != m_bs.DataOffset)
				continue;
			else
				return kNeedMore;
		default:
			break;
		}

		if (MFX_ERR_NONE == s && syncp)
		{
			s = MFXVideoCORE_SyncOperation(m_session, syncp, 60000);

			if (MFX_ERR_NONE != s)
				DPRINTF_HIGH("MFXVideoCORE_SyncOperation %s", MFXEngine::status2str(s));

			if (MFX_ERR_NONE == s && !m_bs.DataOffset)
				DPRINTF_HIGH("@@@ MFX_ERR_NONE sample size %d/%d", m_bs.DataOffset, sample.size);

			if (!m_bs.DataOffset) continue; // hack!?

			break;
		}

...

I put some additional debug output in the codepath. Here we go:

[55524] Trying to open texture 1072x448 0x40007000
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/523
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/523
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/523
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/523
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/523
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/523
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/523
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/523
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/523
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/523
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/523
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/523
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/2358
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/21374
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/17304
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/54540
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/54540
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/54540
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/54540
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/31990
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/26458
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/28043
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/30178
[55824] [0x0000d748][MFXLiveVideoRenderer::feed] @@@ MFX_ERR_NONE sample size 0/37165

sample.size is the actual size of the supplied buffer.

As you can see sometimes it needs several retries (for example that 523 bytes sample took 12 retries, 54540 took 4 retries and so on).

[Nina_K_Intel]

Hi Anton,

Actually, you should not care for the DataOffset, decoder has internal buffering and it is not strictly governed how it reads/takes data from input bitstream. What is governed - when SyncOperation returned ERR_NONE the application can access the surface_out data and it must be a valid decoded frame. If you see the data is not valid - can you please prepare a reproducer based on either simple_2_decode tutorial or sample_decode sample and share the input stream with us as well? We will also need the details of your system - OS, platform, graphics driver version, Media SDK version.

But as a first step I suggest you change this place, might be that the problem is here: 

if (sample.size != m_bs.DataOffset)

19

continue;

If MFX_ERR_MORE_DATA is returned the app must read more data to the input bitstream, independent of the state of bitstream params. Can you please try this and let me know?

Regards,

Nina

[Anton_Z_]

Nina Kurina (Intel) wrote:

if (sample.size != m_bs.DataOffset)

19

                continue;

If MFX_ERR_MORE_DATA is returned the app must read more data to the input bitstream, independent of the state of bitstream params. Can you please try this and let me know?

Regards,

Nina

This could lead to some redesign of my decoding loop, so its a matter of time... Anyway, if the problem exists i guess i am stuck infinite in the while(true) loop, isnt it? I mean if 'continue' is triggered i just retry MFXVideoDECODE_DecodeFrameAsync with same data and got the same result? But this never happends.

[Nina_K_Intel]

Hi Anton,

Not necessarily stuck forever as decoder has an internal state which may change from one call to another. But these are all just speculations, especially as I don't see your full code. I strongly suggest to provide us a reproducer for the issue (non-valid data in the decoded surface after ERR_NONE from SyncOperation would be an issue, non-intuitive DataOffset changes alone are not considered an issue), based on one of the sample apps. Then we will certainly investigate!

Regards,

Nina