Intel® Integrated Performance Primitives
Deliberate problems developing high-performance vision, signal, security, and storage applications.
6704 Discussions

Regarding H264 encoding for a DirectShow - IPP application

sasishiju
Beginner
751 Views

Hi,

Please give me clarification on the below for my web cam capture and send over internet application?

1. I read that the UMC's H264 encoder needs YUV input buffer in the GetFrame method. What kind of YUV is needed out of the commonly known formats such as I420, YV12, UYVY, YVYU etc?

2. I referred the EncodeStream method towards the end of IPP UMC reference manual. It seems to encode a number of frames given as a parameter from the input YUV buffer. Is this always the correct approach? Can we encode each buffer by buffer as captured by the Webcam?

In other words, is it advantageous to construct an big chunk of input buffer and encode in one shot instead of encoding each frame by frame and send over internet?

3. There is one bit rate parameter to be given in the H264EncoderParams.info.bitrate variable. What is the value that need to be filled in here? I have a guess - Is it calculated as the frame size * bits per pixel*frame rate? Please confirm.

0 Kudos
12 Replies
Chao_Y_Intel
Moderator
751 Views

Hi,

1) It is I420. Check the article here (YUV(I420) and Y, U, V separate plane part ) on other format. http://software.intel.com/en-us/articles/getting-started-with-intel-ipp-unified-media-classes-sample/

2) The application need to use GetFrame() method to encode one frame data. If you have a buffer data, it needs to encode frame by frames.

3) The bit rate parameter is to suggest the encoder the targeted encoded video size. It is bits per seconds.


Thanks,
Chao

0 Kudos
sasishiju
Beginner
751 Views

Hi,

1) It is I420. Check the article here (YUV(I420) and Y, U, V separate plane part ) on other format. http://software.intel.com/en-us/articles/getting-started-with-intel-ipp-unified-media-classes-sample/

2) The application need to use GetFrame() method to encode one frame data. If you have a buffer data, it needs to encode frame by frames.

3) The bit rate parameter is to suggest the encoder the targeted encoded video size. It is bits per seconds.


Thanks,
Chao

Hi Chao,
Thanks for your reply.

1)I referred to the section about I420 as given in the link.But the code samples initialize the Media Data objects using UMC::YV12. Aren't I420 and YV12 different? Please confirm.

3)In the case of 3,

a)Is there any relationship between the number of bits present in the input frame and the Bitrate value? If yes, what is the relation?

b)In case of no relation, how can an application decide on this parameter to be set? Any guideline or formula as such?

Thanks in advance,
Shiju
0 Kudos
Chao_Y_Intel
Moderator
751 Views

Shiju,

For the first question: In I420/YV12 format, the U,V plan order is different. I420 takes Y/U/V order, and YV12 takes Y/V/U order. The input format for media Data is decided by your input YUV data. If your input YUV data is YV12, it can set as YV12 format ( Many YUV files are stored at that format, so that simple sample code takes YV12 as the format), or the video is I420, it needs to set it as UMC::YUV420. The codec internally will convert it into UMC::YUV420 format to encode.

As to the bitrate, this is decided by your application, the encoder will use the rate control method to match the encoded video to that bitrate. Of course, the higher bit rate can achieve higher video quality, but the encoded file size (or required network bandwidth if transferred in network) will increase. You can did some test to find what is best for your application.

Thanks,
Chao

0 Kudos
sasishiju
Beginner
751 Views

Shiju,

For the first question: In I420/YV12 format, the U,V plan order is different. I420 takes Y/U/V order, and YV12 takes Y/V/U order. The input format for media Data is decided by your input YUV data. If your input YUV data is YV12, it can set as YV12 format ( Many YUV files are stored at that format, so that simple sample code takes YV12 as the format), or the video is I420, it needs to set it as UMC::YUV420. The codec internally will convert it into UMC::YUV420 format to encode.

As to the bitrate, this is decided by your application, the encoder will use the rate control method to match the encoded video to that bitrate. Of course, the higher bit rate can achieve higher video quality, but the encoded file size (or required network bandwidth if transferred in network) will increase. You can did some test to find what is best for your application.

Thanks,
Chao

Hi Chao,
Thank you! I have a few more queries.

1. Can you please let me know if there is any format other than I420 and YV12 ? If there are more, please direct me to the the complete list of input formats supported by H264 encoder?

2. In the example program given, we have a line like
Params.key_frame_controls.method=1; What is this meant for? What are the possible values for method field? Is it a compulsory field?

3.In the reference PDF document, we have a huge list of parameters for the UMC::H264EncoderParams such as coding_type, B_frame_rate, treat_B_as_reference, num_ref_frames, num_ref_to_start_code_B_slice etc etc.Are these meant to be filled in by the application program? Could you please let me know the set of parameters that need to be set by the program?

4. My starting point in the application is the Webcam captured data. Is it feasible from my part to devise a solution which sends Key Frame, followed by difference frames, followed by Key Frame followed by difference frames etc? This I know, doesn't relate to IPP as such, but please help if you can.

Thanks in advance,
Shiju

0 Kudos
sasishiju
Beginner
751 Views
Quoting sasishiju

Shiju,

For the first question: In I420/YV12 format, the U,V plan order is different. I420 takes Y/U/V order, and YV12 takes Y/V/U order. The input format for media Data is decided by your input YUV data. If your input YUV data is YV12, it can set as YV12 format ( Many YUV files are stored at that format, so that simple sample code takes YV12 as the format), or the video is I420, it needs to set it as UMC::YUV420. The codec internally will convert it into UMC::YUV420 format to encode.

As to the bitrate, this is decided by your application, the encoder will use the rate control method to match the encoded video to that bitrate. Of course, the higher bit rate can achieve higher video quality, but the encoded file size (or required network bandwidth if transferred in network) will increase. You can did some test to find what is best for your application.

Thanks,
Chao

Hi Chao,
Thank you! I have a few more queries.

1. Can you please let me know if there is any format other than I420 and YV12 ? If there are more, please direct me to the the complete list of input formats supported by H264 encoder?

2. In the example program given, we have a line like
Params.key_frame_controls.method=1; What is this meant for? What are the possible values for method field? Is it a compulsory field?

3.In the reference PDF document, we have a huge list of parameters for the UMC::H264EncoderParams such as coding_type, B_frame_rate, treat_B_as_reference, num_ref_frames, num_ref_to_start_code_B_slice etc etc.Are these meant to be filled in by the application program? Could you please let me know the set of parameters that need to be set by the program?

4. My starting point in the application is the Webcam captured data. Is it feasible from my part to devise a solution which sends Key Frame, followed by difference frames, followed by Key Frame followed by difference frames etc? This I know, doesn't relate to IPP as such, but please help if you can.

Thanks in advance,
Shiju


Hi Chao, could you please get back to me on the above queries so that I can proceed further?

0 Kudos
Chao_Y_Intel
Moderator
751 Views

1> You can check the following format at umc_structors.h.

enum ColorFormat

{ ...}

Most of them can be used as the input video data. Note, if it is not the YUV420, the codec will convert the color first.

2). This value can be 0 or 1. If is 0, the codec will set the key frame interval. If it is 1, you can specify the interval. For example, every 10 frames have 1 I frame. You can does not set it, and let the encoder choose the value.

3) If you are not familiar with them, you can only set some required values, and use default value for others.

Params.info.clip_info.height=imgHeight;

Params.info.clip_info.width=imgWidth;

Params.info.bitrate = ??;

Check the following two examples, if you want to set them to control encoder behavior.

http://software.intel.com/en-us/articles/setting-h264-encoding-parameters-in-intel-ipp-media-processing-samples/

4)I am not sure if I fully understand your question. The encoder will create some key frames (I frames), and P/B frames. I think your application need to implement how to sends/ receives these frame data.

Thanks,
Chao

0 Kudos
sasishiju
Beginner
751 Views
Hi Chao,
Thank you very much for the help. I will try out my options and if needed, will get back to you.

With best regards,
Shiju
0 Kudos
sasishiju
Beginner
751 Views
Quoting sasishiju
Hi Chao,
Thank you very much for the help. I will try out my options and if needed, will get back to you.

With best regards,
Shiju

Hi Chao,
One doubt. The UMC manual says "

The Unified Media Classes are intended for use with various decoding, encoding, and

transcoding applications, as well as media plug-ins and filters, for example, Microsoft

DirectShow* filters. "

My question is, if there is any special support in IPP for using with the DirectShow applications?

0 Kudos
Tamer_Assad
Innovator
751 Views

Hi Shiju,

You just need to implement your DirectShow transform filter, and call the UMC processing within your filter. Not that you will have to perform some additional work on the output pin of your filter to supply a compatible media type with the downstream filter. For example, the VMR render filter accepts an RGB media type, not YUV.

If you need a ready DirectShow filter, take a look at the Intel Media SDK.

Regards,

Tamer Assad

0 Kudos
sasishiju
Beginner
751 Views
Quoting Tamer Assad

Hi Shiju,

You just need to implement your DirectShow transform filter, and call the UMC processing within your filter. Not that you will have to perform some additional work on the output pin of your filter to supply a compatible media type with the downstream filter. For example, the VMR render filter accepts an RGB media type, not YUV.

If you need a ready DirectShow filter, take a look at the Intel Media SDK.

Regards,

Tamer Assad

Hi Tamer,
I downloaded the Intel media SDK and browsed through the specification PDFs. As you said,I could see the H.264 Encoder DirectShow Filter. Could you please elaborate on thebelow?
1. The Licensingfee and other terms/conditions
2. Is there any hardware compatibility issues with other x86 hardwares where Windows OS ( XP, Vista, Windows 7 etc) will be running?
3. Any exceptional case / known issues in case of non-Intel platforms in the software side? ( We may not be able to enforce the userto use intel hardware only forrunning our software)
4. In our programs, we make use of MEDIATYPE_Video as major type for query/set methods. Could you please let us know what is FORMAT_VideoInfo2 ?

Thanks,
Shiju

0 Kudos
Chao_Y_Intel
Moderator
751 Views

Shiju,

here is the support forum for Intel Media SDK, you can ask the problem there, and more expert can help you there:
http://software.intel.com/en-us/forums/intel-media-sdk/

Thanks,
Chao
0 Kudos
sasishiju
Beginner
751 Views
Hi,
I have one doubt in H264 encoding usingUMC::H264VideoEncoder. By reading through the H 264 documents, i could see that the standard proposes space/time efficient encoding by using Key Frames and difference frames.
Could you let me know if there is any such support in the IPP - UMC classes? Otherwise, frame by frame encoding may not be a great idea in our application.
Thanks in advance,
Shiju
0 Kudos
Reply