Crash in H264 decoder when missing the first slice of the first IDR

[Emmanuel_W_]

Hi,

I am using version 6.1.5.068 of the samples.

It seems that there is a crash in the H.264 decoder when the first IDR slice of an IDR access unit is missing.
From within the task supplier, RestorErrorRect is called on the H264SegmentDecoderMultiThreaded.
The later function crashes because m_SD was never created.

I was able to fix the issue by creating the SegmentDecoderHPBase if it is null in that function (note that because the m_pSeqParamSet and m_pCurrentFrame are not set yet I ended using the pCurrentFrame of the slice beeing passed in and a hardcoded profile_idc when calling the CreateSD function.

I have also noticed that once RestoreErrorRect is called, the frame is marked as non existing. Is there any reasons to treat a concealed frame any different ?

[Chao_Y_Intel]

Hello,

Thanks for the report. I am forwarding it to the engineer owner about the problem.

Thanks,
Chao

[Chao_Y_Intel]

Hi,

Here some comments from our engineering owners:

we need to check and create m_SD if it is nessesary. Well fix it in followingreleases.

>I have also noticed that once RestoreErrorRect is called, the frame is marked as non existing. Is there any reasons to treat a concealed frame any different ?

Yes, 6.1 gold decoder marks frame as non-existing. This frame is not for display, but can be used as reference. Since 7.0 beta version of IPP decoder is just set error flag, so application can decide if it wants to display this frame or not.

Thanks,
Chao

[j_miles]

Hi guys,

It would be beneficial if you (or eweber) could post the details on the fix (of creating m_SD) here. Thanks.

All that is needed for replicating it, is that just removing/discarding the first slice of a (multi-slice) IDR picture? Or are there any other special requirements for the stream?

Thanks

- Jay

[Emmanuel_W_]

Hi,

You can reproduce the issue by discarding the first IDR slice of a multi slice IDR.

Here is the patch I have done on my side:

[bash]void H264SegmentDecoderMultiThreaded::RestoreErrorRect(Ipp32s startMb, Ipp32s endMb, H264Slice * pSlice)
{
    m_pSlice = pSlice;

    if (startMb >= endMb || !m_pSlice || m_pSlice->IsSliceGroups())
        return;

    try
    {
        H264DecoderFrame * pCurrentFrame = m_pSlice->GetCurrentFrame();

        if (!pCurrentFrame)
        {
            VM_ASSERT(false);
            return;
        }

        H264DecoderFrame * pRefFrame = pCurrentFrame->GetRefPicList(m_pSlice->GetSliceNum(), 0)->m_RefPicList[0];

        pCurrentFrame->SetFrameExistFlag(true); // mark concealed frame as existing otherwise they will not be used
                                                // for display
        pCurrentFrame->SetError(1);

        if (!pRefFrame || pRefFrame->IsSkipped())
        {
            pRefFrame = m_pTaskBroker->m_pTaskSupplier->GetDPBList()->FindClosest(pCurrentFrame);
        }

        // create a segment decoder now if for anyreason we do not have one. 
        if(!m_SD)
        {
            m_SD = CreateSD(
                    bit_depth_luma,
                    bit_depth_chroma,
                    pCurrentFrame->m_PictureStructureForDec < FRM_STRUCTURE,
                    pCurrentFrame->m_chroma_format,
                    true);
            VM_ASSERT(m_SD);
        }
        m_SD->RestoreErrorRect(startMb, endMb, pRefFrame, this);
    } catch (...)
    {
        // nothing to do
    }
}
[/bash]

Emmanuel

[j_miles]

Hi Emmanuel,

Great. Thanks for the tip! I was able to reproduce with a simple test just removing an the first IDR slice of a multi-slice IDR picture.
You may want to go with the actual level-id of the slice from the Sequence Parameter Set associated with the slice (which must be the same associated with the missing slice) instead of forcing true for high-profile, i.e. something similar to this:

[cpp]        if (!m_SD)
        {
            if (!m_pSlice->GetSeqParam())
            {
                VM_ASSERT(false);
                return;
            }
            m_SD = CreateSD(
                          bit_depth_luma,
                          bit_depth_chroma,
                          pCurrentFrame->m_PictureStructureForDec < FRM_STRUCTURE,
                          pCurrentFrame->m_chroma_format,
                          100 <= m_pSlice->GetSeqParam()->profile_idc);
        }
        VM_ASSERT(m_SD);[/cpp]

And for completeness and reference for others, one must also initialize the m_SD member variable to NULL at class initialization (when do Intel/IPP learn this..?).

While looking at this, I noticed something that at first eye-sight looked worrisome. I have not had time to fully analyze it through all levels of templates but I wanted to share my thoughts: The different Segment Decoders are initialized as static (local) members. I can not really see the point in that. On the other hand, it may have a potential of creating problems if using multiple H.264 decoder instances from multiple threads (on a multi-core processor). The specific thing that caught my eye is the check on 'global_sds_array[0][0] == 0' in the static CreateSoftSegmentDecoder function of the CreateSegmentDecoderWrapper template class. Any comments (e.g. can someone assure me that this will work in any situation)...?


- Jay

[Chao_Y_Intel]

Jay,

thanks for the feedback.
For the question:
>While looking at this, I noticed something that at first eye-sight looked worrisome. I have not had time to fully analyze it through all levels of templates but I wanted to share my thoughts: The different Segment Decoders are initialized as static (local) members. I can not really see the point in that. On the other hand, it may have a potential of creating problems if using multiple H.264 decoder instances from multiple threads (on a multi-core processor). The specific thing that caught my eye is the check on 'global_sds_array[0][0] == 0' in the static CreateSoftSegmentDecoder function of the CreateSegmentDecoderWrapper template class. Any comments (e.g. can someone assure me that this will work in any situation)...?


All Segment Decoders are initialized once in constructor of global object SDInitializer. This will work in any situation except DLL with own entry point (In this case all UMC global variables willnt be initialized).

Thanks,
Chao

[Emmanuel_W_]

Thanks for the update Jay,

I think I started by doing something around this line but was getting into a scenario where m_pSlice was not available. I can't remember exactly how this happens but I will try to reproduce.

Emmanuel