I utilize the h.264 encoder to encode the video and send to decoder for play via socket protocol.
For fast decode in our real-time application, I try to tune some parameters (such as entropy coding method, the number of reference frames, rate control method and their related parameter... etc), but the playing fps of decoded video is not improved. The improved of video quality is more obvious than fps at any parameter set in same experimental environment.
Some literatures in this forum are good for my requirement: 1. How to use h.264 High Profile but disable CABAC and B slices
2. CAVLC Enable
3. Problem to decode H264 video in real-time
My h.264 parameter has set the high profile, enable CAVLC and disable B slices to test, but the decoding speed is not faster than enable CABAC and B slices. Why is it so If I tune the rate control method as CQP in different QP value, only the quality is difference.
Could you please elaborate on how you measure the decoding speed? If you look at playabck fps it could be limited by renderer to correspond to time stamps (stream frame rate).
For the run-time error case, there is a helpful tool in Media SDK 2012 package under \tools, it is called Media SDK Tracer. You can use it to get a log file of all the MSDKconfiguration parameters and calls happenning during the problem run. We can try to analyze your issue using such log if you share it. Please don't forget to check "Per-frame logging", it could be helpful.
In our definition, the FPS is denoting the frame per second in playing decoded video. For our real-time application, some overdue frame will be droped and not played, because it will increase the latency time for playing video. If the decoding time is longer, the latency time of playing video would be increased and more overdue frame would be generated. In other words, If the compressed streaming is lower or the the decoder time can be faster, the FPS would be improved.
Thus, in order to reduce the compressed streaming, we try to tune the parameter of h.264 encoder for decoding faster, and some literatures is presented how to adjust. But the FPS is not improved more even unchanged, only the improved video quality is more obvious. Theoretically, when only the compressed streaming is lower in same test environment and bandwidth, the decoding time would be shorter but the video quality is worse with QP greater or limited bit-rate lower in CQP and VBR mode.
The decoder side is applied the FFMPEG h.264 decoder, and we don't modify any source code. When we utilize the x264 encoder in same decoder, the decoding result is right with our adjustment. I can not understand why the decoding result is not meet the above theory in intel media h.264 encoder?
Am I understanding correctly that your goal is to create a stream which 1) can be decoded faster 2) can be transferred over the network faster (i.e. has smaller size)?
Then - you try same settings (no B frames, no CABAC) using x264 and Intel MSDK and x264 allows to increase fps (can you tell fps values without special settings and with them?) but with MSDK encoder settings have no effect on decoding fps?
If I got it right, I have several suggestions:
try enable MaxDecFrameBuffering = 1. This should tell decoder to reduce internal buffering and improve decoding latency (but not speed)
set Encoder.mfxVideoParam.AsyncDepth = 1. This way encoder will work in synchronous mode. Call SyncOperation after every EncodeFrameAsync call
verify that the settings you provided to MSDK encoder were actually used in encoding (call Encoder::GetVideoParam, analyze the output stream)
MSDK Tracer log for encoding part could help us understand what is wrong (please share)
compare SPSPPS headers of MSDK and x264 encoded streams
would be helpful if you shared those 2 encoded streams with us so we can analyze too. Please follow the instructions:
I think that our questions are solved and the decoding speed are raised by setting CAVLC, but one issue was found by my test carelessly. More run-time errors were occurred in Win7 platform but in WinXP at Visual Studio 2005, and the tracer tool can not execute in WinXP but in Win 7.
Media SDK doesn't officially support WinXP. Thought it should work quite fine, if you don't use Direct3D9 surfaces in your application. Can you make sure that only system memory is used as input/output for MSDK functions? If this is not the source of the problem, could you give more details on the run-time errors you observe - which MSDK function triggers the error, what is the error status, any message etc?