How to optimize UMC::H264VideoDecoder to decode multiple video streams

[Hyunho_Kim]

Hello,

I'm developing a s/w for monitoring h264 video streams from the ip cameras.

My company is using MainConcept H264 Decoder now, but we have a plan to change video decoder to Intel IPP. (So we already purchased 2copy of IPP)

In short, I made own decoder class using UMC of IPP and MainConcept function to compare performance of decoding time and tested it. As a result, Mainconept is faster than UMC::H264VideoDecoder of IPP. (more than 2times)

TestCase is..

1. Application make threads per each video streams (It can receive streams from 16 ip cameras, so there are 16 threads for decoding in a application )

2. A thread have it's own video decoder object.

3. A thread decode a camera's h264 video data and then push decoded video into rendering queue.(or no rendering)

4. Application has been linked as dynamic linked library. (both IPP and Mainconcept)

5. I tested it with 16 channel video decoding, 30fps, Wide D1(928*480) for 10 seconds.

6. Mainconcept take about 1 seconds, IPP take more than 2 seconds.

I think I can improve decoding performace of IPP by modification of UMC class, but I don't know how to change it. 

Please give me some advices about UMC class or IPP.

Best regards.

See below for understanding my code (It decode one video frame, it can decode h264, mpeg4 and mjpeg)
int IppCommonVideoDecoderProduct::Decode(const XNSDECODER_VIDEO_INFO &video_info, XNSDECODER_VIDEO_BUFFER *video_buffer)
{
UMC::Status status;
UMC::MediaData data_in;
UMC::VideoData data_out;

if (video_decoder_params_->m_info.videoInfo.m_iWidth != video_info.frame_width ||
video_decoder_params_->m_info.videoInfo.m_iHeight != video_info.frame_height) {
video_decoder_params_->m_info.videoInfo.m_iWidth = video_info.frame_width;
video_decoder_params_->m_info.videoInfo.m_iHeight = video_info.frame_height;
if ((status=video_decoder_->Init(video_decoder_params_.get())) != UMC::UMC_OK) {
return XNSDECODER_ERROR_DECODE_FAIL;
}
}

data_in.SetBufferPointer(video_info.frame_data, video_info.frame_size);
data_in.SetDataSize(video_info.frame_size);

int error_code = SetOutData(video_info, video_buffer, &data_out);
if (error_code != XNSDECODER_SUCCESS) {
return error_code;
}

if (deinterace_option_ != XNSDECODER_DEINTERACE_OFF) {
data_out.m_picStructure = UMC::PS_PROGRESSIVE;
}

if ((status = video_decoder_->GetFrame(&data_in, &data_out)) != UMC::UMC_OK) {
// h.264 i-frame decoding error
if (status == UMC::UMC_ERR_NOT_ENOUGH_DATA) {
status = video_decoder_->GetFrame(NULL, &data_out);
}
}
if (status != UMC::UMC_OK) {
return XNSDECODER_ERROR_DECODE_FAIL;
}

return XNSDECODER_SUCCESS;
}

[Pavel_V_Intel]

Good day. Do you plan to use internal decoder threading? Currently you force decoder to extract frame on each loop, this can significantly decrease internal threading performance. If you plan to stick to this way it is better to limit threads to 1 during initialization. video_decoder_params_->m_iThreads = 1; I recommend to use more asynchronous way (by not forcing decoder to return frames), this will allow you to get balance between internal and external threading to use resources more effectively. Also as I see from screenshot you use resizing a lot. I don't know how you resize frames currently, but It could be more effective to use UMC post-processor than external filters: VideoResizeParams vrParams; VideoProcessing videoProc; videoProc.AddFilter(FILTER_RESIZER, (BaseCodecParams*)&vrParams); pVideoDecParams->m_pPostProcessor = &videoProc;

[Hyunho_Kim]

Hello Pavel, It was nice to receive your mail this morning. Could you tell me more specific about more asynchronous way? Currently, decoding method in my company is all this way, so I don't know well about asynchronous way. I already have set m_iThreads to 1 in Initialize() process and I think resize is only for rendering performance not decoding. I didn't calculate rendering time on the test. Thank you for your help. Best regards, Hyunho

[Pavel_V_Intel]

You can feed decoder with input data until it returns a frame for itself; this way you can utilize internal threading. [cpp] if(end_of_stream) { status = video_decoder_->GetFrame(NULL, &data_out); if (status == UMC::UMC_ERR_NOT_ENOUGH_DATA) return END_STATUS; } else { status = video_decoder_->GetFrame(&data_in, &data_out); if (status == UMC::UMC_ERR_NOT_ENOUGH_DATA) return GIVE_NEXT_FRAME_STATUS; } if(status != UMC_OK) return XNSDECODER_ERROR_DECODE_FAIL; return XNSDECODER_SUCCESS; [/cpp]

[Jimmy_S_1]

The video decoder is a device or software that enables compression or decompression of digital video. The compression is usually lossy. Historically, video was stored as an analog signal on magnetic tape. Around the time when the compact disc entered the market as a digital-format replacement for analog audio, it became feasible to also begin storing and using video in digital form, and a variety of such technologies began to emerge.