Intel® Integrated Performance Primitives
Deliberate problems developing high-performance vision, signal, security, and storage applications.

H264 decoder problem

selea
Beginner
1,979 Views
Hi all. I'm using IPP lib version 7.0.4 with the latest IPP sample7.0.4.054
I have got a crash with the H264 decoder when I'm decoding a 1280x720 stream. Not all the HD ready stream get this problem, but this sistematically crash.
I attach the stream that cause me the problem.
I write the stack trace that cause me the problem:
void H264SegmentDecoder::DeblockMacroblockPSlice()
void H264SegmentDecoder::PrepareDeblockingParametersPSlice()
void H264SegmentDecoder::PrepareDeblockingParametersPSlice16x16Vert()
inline size_t H264SegmentDecoder::GetReferencePSlice(Ipp32s iMBNum, Ipp32s iBlockNum)
iMBNum=1840
iBlockNum=0
size_t DeblockPicID(Ipp32s index)
index=0
return m_ID[index]; //this row cause the problem
it's seems thatm_ID is not null but is in an inconsistent state
Can you reproduce the problem?
0 Kudos
1 Solution
Ying_H_Intel
Employee
1,980 Views

Hi Selea,

Glad to know you make progress. and thank you very much to share your experience here.

You are exact right, the GetFrame() is not always produce a frame. There is some data remain in buffer.I guessthismay be the most importantpointwhendevelop H.264 decoderwith stream data.We may need take care of the loop and condition in the loop very carefully.

See the explanation in the UMC docu

Syntax virtual UMC::Status GetFrame(UMC::MediaData* pSrc, UMC::MediaData* pDst)
Description
The method processes data.
Ifthe user allocates input and output buffers. If GetFrame does
not use all data from the input buffer, the method updates data pointer, value of the actual
data in the buffer and start time.

If the codec uses internal frame buffering it can return UMC::UMC_ERR_NOT_ENOUGH_DATA
at the beginning of the processing. This error code means that the input data is buffered
but the output data is not ready to be delivered. The application has to continue to call
GetFrame with the next input data.

To retrieve buffered data at the end of the processing, the application should call GetFrame
with NULL input pointer while output is not empty.

(and ifwe did search, there aremany discussion about the h.264 decoder's delay or get UMC_ERR_NOT _ENOUGH_DATA in the forum.Unfortunately whentheapplication become large and complex, the basic problem weresubmerged)

With comment if (/*(3 > in->GetDataSize()) &&*/ andif (UMC_ERR_NOT_ENOUGH_DATA == ret)
{
if (bEndOfStream)
break;
// else
// continue;
}

//if (UMC_OK != ret)
// continue;
numDecodedFrames++;
printf("Frame decoder %d\n", numDecodedFrames);

I can't reproduce the crash, butI believedyou will getwrong result if discard someremain data by some condition clause.
For example, with above comment,iwill get 27 frame and thereal frame with thestream is 24 frame. some frame is actually black.

Best Regards,
Ying

View solution in original post

0 Kudos
30 Replies
selea
Beginner
423 Views
try to decode in 2 separeted thread.I'm on Window ia32UMC sample7.0.4.054 too, and I'm using static lib version of the IPP
0 Kudos
Ying_H_Intel
Employee
423 Views
Hi Selea,

if with
DataIn.SetBufferPointer(cVideoData,VideoDataSize);
DataIn.SetDataSize(VideoDataSize);

Params.m_pData = &DataIn; Params.lFlags=0; Params.numThreads=4;
if(status = H264Decoder.Init(&Params)!=UMC::UMC_OK)
return;

H264Decoder.GetInfo(&Params);
imgWidth=Params.info.clip_info.width; imgHeight=Params.info.clip_info.height;
it works fine.
how you to decode in 2 separeted thread? any detials?

Regards,
Ying
0 Kudos
selea
Beginner
423 Views
No, I mean to decode multiple streams in separated thread: for example 2 threads that decode the h264 stream.
set the Params.numThreads=1 and try to create 2 thread that decode the stream...anyway I can't understand why you can decompress without problem.
Have you used the static IPP library version?
0 Kudos
Ying_H_Intel
Employee
423 Views
Hi Selea,

Yes, I have tried static IPP library (both sequential(ipp*_l.lib) or parallel (ipp*_t.lib)).They can decompressyour stream fine.

So the problemlooks bemultiple threading. How do you create 2 threads by Win32 Windows API?Could you provide methe simple test code?

(if it is confidential, you may use private post).

Regards,
Ying
0 Kudos
selea
Beginner
423 Views

Usually when I have to create 2 thread I use the function _beginthreadex.So you can create a function to decode the stream, create 2 thread with _beginthreadex that execute this function and run it at the same time. Try and let me know. Anyway I can reproduce the problem decoding even in 1 thread. Have you tried with the latest lib and sample?


0 Kudos
selea
Beginner
423 Views

After some days of try with the umc_h264_dec_con sample and my application I found the problem.

Was in my application, so I apologize with youYing to waste your time.

Anyway, I explain what was the problem.

Usually we read the H264 stream from the network or from an mkv video file: instead the samples works on a h264 bitstream. I make a H264->GetFrame on every frame I got, but I always consider that the decoder consume all the video data of the frame (usually it's so). Anyway in this particular case I got some frame that are not totally consumed but got me a complete image. For example I have a frame read from a the stream of 35kb and after GetFrame I have consumed only 15kb. I did not check the reamain bytes that now I put in the decoder again.

So probably after I wrong and discard the remain data without fill it in the GetFrame, next frame cause me a crash of the decoder.

Was my error, but I think you can check why the decoder crash.

To simulate the decoder crash change the sample code like this:

from

if ((3 > in->GetDataSize()) &&

to

if (/*(3 > in->GetDataSize()) &&*/

and comment out the continue; after initialization

try that with the 1280x720.h264 I attached and you can get the exception

let me know

0 Kudos
Ying_H_Intel
Employee
1,981 Views

Hi Selea,

Glad to know you make progress. and thank you very much to share your experience here.

You are exact right, the GetFrame() is not always produce a frame. There is some data remain in buffer.I guessthismay be the most importantpointwhendevelop H.264 decoderwith stream data.We may need take care of the loop and condition in the loop very carefully.

See the explanation in the UMC docu

Syntax virtual UMC::Status GetFrame(UMC::MediaData* pSrc, UMC::MediaData* pDst)
Description
The method processes data.
Ifthe user allocates input and output buffers. If GetFrame does
not use all data from the input buffer, the method updates data pointer, value of the actual
data in the buffer and start time.

If the codec uses internal frame buffering it can return UMC::UMC_ERR_NOT_ENOUGH_DATA
at the beginning of the processing. This error code means that the input data is buffered
but the output data is not ready to be delivered. The application has to continue to call
GetFrame with the next input data.

To retrieve buffered data at the end of the processing, the application should call GetFrame
with NULL input pointer while output is not empty.

(and ifwe did search, there aremany discussion about the h.264 decoder's delay or get UMC_ERR_NOT _ENOUGH_DATA in the forum.Unfortunately whentheapplication become large and complex, the basic problem weresubmerged)

With comment if (/*(3 > in->GetDataSize()) &&*/ andif (UMC_ERR_NOT_ENOUGH_DATA == ret)
{
if (bEndOfStream)
break;
// else
// continue;
}

//if (UMC_OK != ret)
// continue;
numDecodedFrames++;
printf("Frame decoder %d\n", numDecodedFrames);

I can't reproduce the crash, butI believedyou will getwrong result if discard someremain data by some condition clause.
For example, with above comment,iwill get 27 frame and thereal frame with thestream is 24 frame. some frame is actually black.

Best Regards,
Ying

0 Kudos
selea
Beginner
423 Views
thanks Ying, my application already take care of theUMC_ERR_NOT _ENOUGH_DATA that's ok. The only problem was that I never saw in all the h264 stream before a behaviour like that. So I never see before the GetFrame consume only some bytes of the input buffer. Now I got it and I fix it.
Anyway with the code you modify, if you look at the VisualStudio output you will see an exception throw at the line
size_t DeblockPicID(Ipp32s index)
{
return m_ID[index];
}
That cause me the crash before, but looking at the code I see the exception is caught correctly, so it's ok.
Sorry again to waste your time, but the problem was that I did not see that "bytes consuming behaviour" before. Probably it's a difficult condition to obtain in a live stream that come from a camera.
0 Kudos
j_miles
Beginner
423 Views
Now, it is unclear why there is an exception thrown at the lines given but I suspect it could be an access violation or similar due to out-of-bounds indexing. Such should never occur! No access violations ever in a decoder library under normal circumstances! (And certainly, they should not be swept under the carpet by merely catching them). So even if the way the decoder has been used by not delivering the right amount of data (or whatever) is not as intended, the decoder should be sufficiently fail-safe in a way that it does not crash. As long as the user does not misuse the library and does provide proper and accessible data, it should be fail-safe even if the user has misunderstood some details in the use. There will obviously be ways that people can misuse a library and those cases will not be able to be handled perfectly and it could even crash then. But this case does not seem like such a case. And the internal DeblockPicID function should obviously always work without access violations!


- Jay
0 Kudos
Reply