Using JPEG encoder with MSFT YUV types

[jnoring1]

Hi,

I'm working on extracting DIBs from Windows Media files using the WMFSDK, and trying to encode them to the jpeg format. I downloaded the JPEG samples provided by the IPP, and have compiled them, and they work well with RGB bitmaps.

However, I want to re-encode video frames that are outputted in either a YV12 or YUY2 format. YV12 is a 4:2:0 planar YUV format, and YUY2 is a 4:2:2 packed YUV format. I tried to modify the encoding classdefined in encoder.cpp (CJPEGEncoder)that came with the JPEG sample to accept these types, but the resulting file is always garbage (vertical pink lines, and a pretty heavy file size).

If I have the WMFSDK output RGB DIBs, the CJPEGEncoder works well, but the goal is to skip the two unneccessary colorspace conversions that occur (once when the WMFSDK outputs RGB, since it does a colorspace conversion from a YCbCr type, and again when the CJPEGEncoder converts it back to a YCbCr type).

A few other questions regarding JPEG compression that I don't understand:

1. I noticed the CJPEGEncoder takes a JCOLOR argument in SetSource that specifies the colorspace of the input image (?). But I also noticed that SetDestination takes a JCOLOR argument as well. I don't understand why a JPEG output image would have a JCOLOR type. I'm sure I'm just missing something here; a clarification would be great.
2. Do the JSS arguments on SetSource and SetDestination refer to the MCU format, or are they associated with the JCOLOR argument? For example, if I submit 4:2:2 YCbCr DIBs, do I need to set this to JS_422?
3. What is the difference between baseline and progressive encodes?

Any help is much appreciated; thanks in advance.

jer

[Vladimir_Dudnik]

Hello Jer

we do not support YUV420 at JPEG encoder side yet (but it is still in our wishlist, so it might be implemented in future versions). But we do support encoding/decoding from/to YUY2 format and that is the reason why we have sampling and color parameters in both SetSource and SetDestination methods. (yes, you need to set sampling to 422 if your DIB is inYCbCr422 format).

JPEG standard provides several compression modes, baseline is when all picture data are compressed in one pass through the data. Progressive mode allow you to compree (and then to decompress) data in several passes through tha data,for example most significant bits first thenless significant. It usually usedwhen you need to transfer image through slow networkchannel, so on client side it could be possible to decompress (and display) rough approximation image first with adding details later on.

Regards,
Vladimir

[jnoring1]

Vladimir,

Thank you for the prompt response. I have a few other questions that I was hoping you could answer for me. I'm still using the IPP version 4.1, so to get the current sample to compile, I had to comment out the following sections of code:

status = ippiYCbCr422_8u_P3C2R(src,srcStep,dst,dstStep,roi);if(ippStsNoErr != status)
{
LOG1("IPP Error: ippiYCbCr422_8u_P3C2R() failed - ",status);return JPEG_INTERNAL_ERROR;
}
...I'm assuming the above function was added after 4.1; is there some combinations of 
functions from IPP 4.1 that I could use to perform the same action (it seems like this 
method is converting the composite 422 data to a planar format?), or would it simply be
easier to upgrade to IPP 5.1?
Thanks again for your help with all of this,
jer

[Vladimir_Dudnik]

You are absolutely correct, this function is intended to convert YUY2 data from pixel-interleaved format to planar format (which is used by JPEG codec internally). This function was added after IPP 4.1. Unfortunately there were no similar function in IPP 4.1 (otherwise what the reason to add this one?). There should not be issues in upgrading to the latest IPP which is 5.1.1 version if you do not use ippm (matrix domain) and ippvc (video coding domain) in your application. In IPP domains mentioned above there were changes in API of some functions (deprecated functions were removed and replaced by new functions with more flexible and power API). In that case you will need to tune your code accordingly. There was a special document to simplify moving from IPP 4.x to IPP 5.x, you can ask it on Intel Premier Support.

[jnoring1]

Thanks again, that's a big help. A few final questions:

What is the m_src.nChannels variable used for? I noticed that in the SetSource call, it takes a parameter specifying the number of channels, but that argument doesn't seem to be used anywhere. My guess is that there is little need for such an argument because # of channels is implicit in the JCOLOR type?

I have a few other options for output types from the WMFSDK, but there's some definite inaccuracies in the MSFT YUV types; they seem to think 411 is the same thing as 420 (e.g. the documentation for the WMFSDK statesthat YV12 is"...YUV video stored in planar 4:2:0 or 4:1:1 format. Identical to I420/IYUV except that the U and V planes are switched.", which is clearly incorrect, 420 != 411).

They types I have at my disposal are:

• WMMEDIASUBTYPE_RGB24
• WMMEDIASUBTYPE_RGB32
• WMMEDIASUBTYPE_I420
• WMMEDIASUBTYPE_IYUV
• WMMEDIASUBTYPE_YV12
• WMMEDIASUBTYPE_YUY2
• WMMEDIASUBTYPE_UYVY
• WMMEDIASUBTYPE_YVYU

...and the documentation isn't totally clear if these are YUV or YCbCr. Are any of these formats, other than the RGB types, compatible with the IPP JPEG library provided by the sample?

[Vladimir_Dudnik]

Yes, currently number of channels is not used at encoder side, but the idea was to add support for JC_UNKNOWN color, which basically should mean - not to do color conversion step. And in this case there is no way to determine number of components in image. So it will be used in future (at least I hope so).

To be honest, using of 411 name for sampling in IPP JPEG codec is not correct as well, it was done so just by historical reasons. The same name was used in IJL since 1998 and so we decided not to change naming. You are right, 420 is not the same as 411.

We did not analized color definitions in Windows Media Photo yet, seems WMMEDIASUBTYPE_YUY2 should be equivalent to ordinary YUY2 format, which is supported by IPP JPEG codec. We also do support RGB24, other formats will require conversion step.

By the way, how do you find performance of WMP codec against JPEG? Is it slower?

In my understanding it should be faster than JPEG2000 at approximately the same image quality level but probably will be a bit slower then JPEG.

Vladimir

[Vladimir_Dudnik]

O-o-p-s, did not realize that you use Windows Media Format SDK, not new image compression codec, proposed by Microsoft. Just disregard related questions.

Vladimir