Got the crash bug with H.264 decoder(Urgent)

[ashoknara1006]

We are purchased the IPP and use IPP 6.1(update 3) and got the large delay with H.264 decoder, we found this topic:http://software.intel.com/en-us/forums/showthread.php?t=67755and modified the UMC source code as Mr.Ying H that wrote on the topic #5 then the delay was avoid, but when we call to GrandStream GXV3140, we encountered the crash issue immediately, the crash point is random, seems memory is corrupted.

The crash is not appears if we do not modify the UMC source code.

Could anyone help me?

Also we have a suggestion: could Intel fix this H.264 delay issue in a official release?

Thanks

The GrandStreamGXV3140 version is:

Product Model:GXV3140

Hardware Version:V0.4A

Software Version:

Boot--1.0.1.2

Program--1.0.3.24

Core--1.0.3.4

GUI-A--1.0.3.3

Base--1.0.3.16

GUI-B--1.0.3.3

DSP--1.0.3.25

[ashoknara1006]

Please help me.

Thanks

[Ying_H_Intel]

Hi Ashoknara,

I recalled there is some risk if modify code like i did, for example, when there is B frame in stream. Could you attached a piece of crashed stream?
Have youtried the suggestion of Jay, how it works?

Or other ideas, for example, reduce the delay by define smallDecFrameBuffer instead of 16, but not avoid the delay completely.

Best Regards,
Ying

[ashoknara1006]

Thank you for your help Ying. I will try to define smallDecFrameBuffer.

I havn't tried with Jay's way because my English is not good, I didn't understand his post clear so far.

I have attached the video stream which received from GrandStream, the format is:

typedef struct {

BYTE byMediaType;

BYTE bKey;

BYTE bEof; // if the maker field of RTP packet header is true, then this bEof is 1, otherwise is 0

DWORD dwSeq; // the seq number from RTP header.

DWORD dwTimeStamp;// this is time stamp from RTP hader

WORD wDataLen; // the RTP payload data length.

}AV_PACKET_HEAD;

I wrote above struct into the file before wrote each RTP packet.

If you want to read the attachment - Test264.dat, you should:

1. read a struct first:AV_PACKET_HEAD

2. then got the rtp payload data size(wDataLen ofAV_PACKET_HEAD)and read it .

3. repeat above steps.

Another thing, does IPP has any plan to avoid the delay in future official release?

Thanks

[Ying_H_Intel]

Hi Ashoknara,

I'm very sorry to tell that no, IPP UMC team haven't plan to avoid the delay in the future official release.

I quotesome comment from our expert:

There is internal buffer which called DPB (Decoded Picture Buffer). By standard, decoder couldn't return frame until DPB is full enough (It'sneeded for correct reordering frames process). So, when there are empty slots in DPB GetFrame function returns

UMC_NOT_ENOUGH_DATA. When DPB is full enough GetFrame return UMC_OK and you can use returned frame.

For using H.264 decoder in video conference applicationsit's recommended to manually adjust DPB buffer size.

Size of DPB depends from level_idc (IPP H.264 encoder parameter) and picture's size.

For example, picture size is 320x280. When you set level_idc to 30 then DPB size will be ~12 frames. In this case, decoder could return first frame to application only after it's received and decode 12 frames. As result, long delay occurs. When you set level_idc to 10, then DPB size will be ~1-2 frames and delay will be short.

Best Regards,
Ying

[ashoknara1006]

Hi Ying, thank you very much for your help.

I have something to suggest, could add a mode in the UMC that easy to use for real time purpose?(don't use hack way), likes video conferencing, Video phone etc.

Thanks

[oxydius]

Hi Ying, the expert comment about H.264 DPB is wrong. The DPB size specified in the standard is a maximum size to allocate, an upper bound on the memory consumption necessary for a given stream. The first H.264 frame has no dependency, the second has one, and the third may have one or two, and so on, but unless they are bidirectional (B-frames), they do not need to be held back and can be returned as they're fed. Frames should only be held when a B-frame is encountered and its dependencies aren't met (as specified by ref_pic_list_reordering() in standard). Only at that point should the DPB fullness approach its maximum.
Another counter-example to the DPB needing to be full (ie. 12 frames) is that whenever an IDR frame is encountered, the DPB should be emptied and all previous reference frames should stop being used.

UMC H.264 decoder always stays at 12 and introduces artificially high latency just so it never has to return UMC_NOT_ENOUGH_DATA, which violates IDR frame behavior. If you decode an IDR frame, you're supposed to get it right away, not 11 frames in the past. This results in people finding seeking hard to implement with UMC and having to either hack it or resort to calling Reset() to forcefully clear the DPB, crashing in the process.

A full DPB makes internal multi-threading much easier to implement so I see the benefit, but if there are no plans to change this, could there at least be an optional setting to force low-latency behavior in the future?

[ashoknara1006]

I quite agree, if the UMC has a optional setting to force low-latency will easy to use for the real time applications.

Hope UMC team consider the suggestion.

[Ying_H_Intel]

Dear All,

Thank a lot for the suggesiton. I have been escalted the feature to the UMC developer team. Hope they may consider it some days.

Digged out one morepossibleworkaroundabout reduce the delay in decoder:

There is VideoDecoderParams::dPlaybackRate parameter (Ipp64f). By default dPlaybackRate equals 1.0. If dPlaybackRate doesn't equal 1.0 (at H.264 decoder Init) then decoder outputs in decoded order instead of display order. It allows to significantly reduce delay.

Best Regards,
Ying