Some questions about H.264

[ganenew]

I utilize the h.264 encoder to encode the video and send to decoder for play via socket protocol.

For fast decode in our real-time application, I try to tune some parameters (such as entropy coding method, the number of reference frames, rate control method and their related parameter... etc), but the playing fps of decoded video is not improved. The improved of video quality is more obvious than fps at any parameter set in same experimental environment.

Some literatures in this forum are good for my requirement:

1. How to use h.264 High Profile but disable CABAC and B slices

2. CAVLC Enable

3. Problem to decode H264 video in real-time


My h.264 parameter has set the high profile, enable CAVLC and disable B slices to test, but the decoding speed is not faster than enable CABAC and B slices. Why is it so If I tune the rate control method as CQP in different QP value, only the quality is difference.

Following is my parameter set

memset(&meco, 0, sizeof(mfxExtCodingOption));
meco.Header.BufferId = MFX_EXTBUFF_CODING_OPTION;
meco.Header.BufferSz = sizeof(mfxExtCodingOption);
meco.CAVLC = MFX_CODINGOPTION_ON;
//meco.MaxDecFrameBuffering = 1;
m_mfxEncParams.ExtParam = new mfxExtBuffer *[1];
m_mfxEncParams.ExtParam[0] = (mfxExtBuffer*)&meco;
m_mfxEncParams.NumExtParam = 1;

m_mfxEncParams.mfx.TargetUsage = MFX_TARGETUSAGE_BEST_SPEED;
m_mfxEncParams.mfx.RateControlMethod = MFX_RATECONTROL_VBR;
m_mfxEncParams.mfx.TargetKbps = targetKbps;
m_mfxEncParams.mfx.MaxKbps = (mfxU16)(1.0*targetKbps);
m_mfxEncParams.mfx.GopOptFlag = MFX_GOP_CLOSED;
m_mfxEncParams.mfx.IdrInterval = frameRate;
m_mfxEncParams.mfx.GopRefDist = 1;
m_mfxEncParams.mfx.GopPicSize = 1 + nPFramePerGOP;
m_mfxEncParams.mfx.NumRefFrame = 2;
m_mfxEncParams.mfx.NumThread = 1;

Another question is trouble for me, If I set the MaxDecFrameBuffering as 1 or NumRefFrame as 1, my visual studio would occur the run-time error in EncodeFrameAsync fuction.

The encoder: Intel media library 2012.
The decoder: FFMpeg v0.1.0 h.264 decoder

[Nina_K_Intel]

Hello,

Could you please elaborate on how you measure the decoding speed? If you look at playabck fps it could be limited by renderer to correspond to time stamps (stream frame rate).

For the run-time error case, there is a helpful tool in Media SDK 2012 package under \tools, it is called Media SDK Tracer. You can use it to get a log file of all the MSDKconfiguration parameters and calls happenning during the problem run. We can try to analyze your issue using such log if you share it. Please don't forget to check "Per-frame logging", it could be helpful.

Thanks,

Nina

[ganenew]

In our definition, the FPS is denoting the frame per second in playing decoded video. For our real-time application, some overdue frame will be droped and not played, because it will increase the latency time for playing video. If the decoding time is longer, the latency time of playing video would be increased and more overdue frame would be generated. In other words, If the compressed streaming is lower or the the decoder time can be faster, the FPS would be improved.

Thus, in order to reduce the compressed streaming, we try to tune the parameter of h.264 encoder for decoding faster, and some literatures is presented how to adjust. But the FPS is not improved more even unchanged, only the improved video quality is more obvious. Theoretically, when only the compressed streaming is lower in same test environment and bandwidth, the decoding time would be shorter but the video quality is worse with QP greater or limited bit-rate lower in CQP and VBR mode.

The decoder side is applied the FFMPEG h.264 decoder, and we don't modify any source code. When we utilize the x264 encoder in same decoder, the decoding result is right with our adjustment. I can not understand why the decoding result is not meet the above theory in intel media h.264 encoder?

[ganenew]

By the way, the frame rate of encoder is set to 25, and the average FPS of playing the decoded video is about 15 and 9 in x264 and intel media sdk in real-time, respectively.

[Nina_K_Intel]

Hello,

Am I understanding correctly that your goal is to create a stream which 1) can be decoded faster 2) can be transferred over the network faster (i.e. has smaller size)?

Then - you try same settings (no B frames, no CABAC) using x264 and Intel MSDK and x264 allows to increase fps (can you tell fps values without special settings and with them?) but with MSDK encoder settings have no effect on decoding fps?

If I got it right, I have several suggestions:

• try enable MaxDecFrameBuffering = 1. This should tell decoder to reduce internal buffering and improve decoding latency (but not speed)
• set Encoder.mfxVideoParam.AsyncDepth = 1. This way encoder will work in synchronous mode. Call SyncOperation after every EncodeFrameAsync call
• verify that the settings you provided to MSDK encoder were actually used in encoding (call Encoder::GetVideoParam, analyze the output stream)
• MSDK Tracer log for encoding part could help us understand what is wrong (please share)
• compare SPSPPS headers of MSDK and x264 encoded streams
• would be helpful if you shared those 2 encoded streams with us so we can analyze too. Please follow the instructions:

1. 1. • - ftp ftp.intel.com
      • - enter 'anonymous' (no quotes of course) aslogin
      • - forpasswordenteryour email address
      • - Don't forget: enter 'bin' to go tobinary mode
      • -cd /pub/incoming
      • - Files are hidden (not visible), so I'll need the_exact_ file name(s)
      • (Note:don't use underscoresin file names)
      • - Files names are case sensitive. So abc.txt, Abc.txt and ABC.txt are three different file names.
      • - Files will remain on ftp server for 24 hours and are then deleted automatically
      • - Files greater than 100MB are not allowed by anonymous users

• There is a white paper on video conferencing/streaming usages of Media SDK which should help you analyze your set up:http://software.intel.com/en-us/articles/video-conferencing-features-of-intel-media-software-development-kit/

Regards,

Nina

[ganenew]

Hello,

I think that our questions are solved and the decoding speed are raised by setting CAVLC, but one issue was found by my test carelessly. More run-time errors were occurred in Win7 platform but in WinXP at Visual Studio 2005, and the tracer tool can not execute in WinXP but in Win 7.

[Nina_K_Intel]

Hi,

Media SDK doesn't officially support WinXP. Thought it should work quite fine, if you don't use Direct3D9 surfaces in your application. Can you make sure that only system memory is used as input/output for MSDK functions? If this is not the source of the problem, could you give more details on the run-time errors you observe - which MSDK function triggers the error, what is the error status, any message etc?

Thanks,

Nina