Multi-channel live-streaming with Intel Media SDK hardware decode.

[James_K_]

Dear all, 

I have few questions about Multi-channel live-streaming with Intel Media SDK hardware decode. 

1.

Following is my implementation code

＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝

// decoder side receive frame from RTSP server frame by frame. Timely Memory copy it to mfxBS structure. 

// Thus, I set async depth to 1 and expect that each round should output one decoded frame at least. 

memcpy_s((void*)(m_mfxBS.Data), m_mfxBS.MaxLength, (void*)(pFrameData->RecFrameBuffer), pFrameData->RecBufSize);

 m_mfxBS.DataLength = pFrameData->RecBufSize;
 m_mfxBS.DataOffset = 0;

 m_mfxBS.DataFlag  = 1;            
 m_mfxBS.TimeStamp = pFrameData->PESInfo.rtpTimestamp;

SyncFrameSurfaces();
 if (!m_pCurrentFreeSurface) 
{// get free surface
      m_pCurrentFreeSurface = m_FreeSurfacesPool.GetSurface();        
 }

 if (!m_pCurrentFreeOutputSurface) 
  {
      m_pCurrentFreeOutputSurface = GetFreeOutputSurface();
 }

do 
    {
        pOutSurface = NULL;
        sts = m_pmfxDEC->DecodeFrameAsync(pBitstream, &(m_pCurrentFreeSurface->frame), &pOutSurface, &(m_pCurrentFreeOutputSurface->syncp));
        
        if (sts == MFX_ERR_NULL_PTR)
        {
                      
        }
        else if (sts == MFX_WRN_DEVICE_BUSY)
        {
            _DPRINTFA("WARNING:MFX_WRN_DEVICE_BUSY DecodeThread(index:%d) device busy!",m_tOptions.dwIndex);
            Sleep(1);
            if (nTryTimes == 5)
                break;
            nTryTimes++;
        }

        if (MFX_ERR_MORE_SURFACE == sts)
        {// MFX_ERR_MORE_SURFACE: somehow decoder need more surface in previous decoded round. 
            // Insert more free surface and set bit-stream to NULL since the bit-stream has been processed finished.
            if (m_pCurrentFreeSurface)
            {
                m_UsedSurfacesPool.AddSurface(m_pCurrentFreeSurface);
                m_pCurrentFreeSurface = NULL;
                pBitstream = NULL;
                SyncFrameSurfaces();
                m_pCurrentFreeSurface = m_FreeSurfacesPool.GetSurface();        
                if (!m_pCurrentFreeSurface)  {
                    _DPRINTFA("DecodeThread(index:%d) Free surface is run out!!", m_tOptions.dwIndex);
                   break;               

                  }

           } 

        else if ((MFX_ERR_NONE == sts) || (MFX_ERR_MORE_DATA == sts)) 
        {// MFX_ERR_NONE: decoder has output decoded frame. break.
            // MFX_ERR_MORE_DATA: decoder need more bit-stream data. break.
            m_UsedSurfacesPool.AddSurface(m_pCurrentFreeSurface);
            m_pCurrentFreeSurface = NULL;
            break;
        }

 // MFX_WRN_VIDEO_PARAM_CHANGED: Get SPS stream data, call decode again to deal with IDR frame behind.    
 } while (MFX_WRN_DEVICE_BUSY == sts || MFX_WRN_VIDEO_PARAM_CHANGED == sts || (MFX_ERR_MORE_SURFACE == sts));

    if (MFX_ERR_NONE == sts && pOutSurface) 
    {// decoding finished!        
        msdkFrameSurface* surface = FindUsedSurface(pOutSurface);
        msdk_atomic_inc16(&(surface->render_lock));
        m_pCurrentFreeOutputSurface->surface = surface;
        m_OutputSurfacesPool.AddSurface(m_pCurrentFreeOutputSurface);
        m_pCurrentFreeOutputSurface = NULL;

        sts = SyncOutputSurface(MSDK_DEC_WAIT_INTERVAL, pFrameData);

    }    

＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝

Unfortunately, sometimes this decode function  would return "MFX_ERR_MORE_SURFACE".（Avg return this error code per two frames)

So that I get one more free surface and call decode function again. 

Somehow the decoder function still return "MFX_ERR_MORE_SURFACE" back till current free surface pool is run out. 

Could you explain more about why "MFX_ERR_MORE_SURFACE"  error code happens? and happens again and again? Thank  you. 

[Sravanthi_K_Intel]

Hello James,

If you allocated enough surfaces (QueryIOSurf can be used here), and you use the pool to find new unlocked surfaces, you should not see this behavior. Surfaces from previous calls get freed up and can be re-used, for which you have to check their lock status when you get free surfaces using your GetSurface() function. 

In particular, you code enclosing "if (MFX_ERR_MORE_SURFACE == sts)" does not look correct to me. As mentioned, use the GetFreeSurfaces() function as described in the manual or sample_decode application. In the sample, we account for the locked field for each surface before picking it for reuse.

If your code does everything in accordance to what is explained above, and it is based off of sample_decode as well - please let us know. Specifically, you may want to send your impl of GetSurface() and also the input stream you are passing (to reproduce the issue).

[James_K_]

Hello SRAVANTHI K. 

Thanks for your quickly reply. 

I did my implementation according to Intel Video Decoding Sample 5.0.461.91752. 

In this sample code, GetSurface() actually can get free surface if it's available after "syncframesurface()" call.

"Syncframesurface" function would dispatch surface to be free or not by its lock count.

 Per QueryIOSurf() function, does it mean that maximum reference frame requested by driver? (H264 dpb size)

Thank you.