Intel® Integrated Performance Primitives
Deliberate problems developing high-performance vision, signal, security, and storage applications.
6704 Discussions

Saving Frames Of Streaming H264 Video Into JPEGs

Stephen_V_1
Novice
2,420 Views

I have streaming H264 video that I want to create jpeg still images of every X frame.  I am using the UMC video decoder example to decode the H.264 video to a "raw" format (I'm using RGB).  Once I want to encode one of the frames into a JPEG, I use the UIC library to copy that  frame of RBG data into a CIppImage object and then encode it using code based off the uic transcode example.

When I save the jpeg to a file and open it, I see the content correctly but the colors are all wrong (mostly shades of pink and green).  I also tried decoding the video frames to RGBA and encode a JPEG from that type, but again the colors are wrong (in this case, the image looks like a negative of the correct colors).  I'm not familiar with image and video processing, so I'm having a hard time determing where something is wrong based upon the results I'm seeing.  Any suggestions of what could be wrong when I have the content correct, just the colors are not? Is there a better way to accomplish my goal rather than using the method I defined in the first paragraph?  Part of my problem is finding a color format that is in common with what the video codec can decode into and what the UIC codec can encode from.  The only options that I see are either RGB or RGBA, does the decoded video frames of RGB type align with the format of RGB data when encoding a JPEG?

Another thing I would like to verify (but can't) is if the decoded frames are correct.  I was unable to find a program that could open a "raw" frame of just the RGB data.  My only guess that the decoded video is correct is that I was able to save the decoded video and then used the umc_video_enc_con sample binary to encode that raw video.  When doing that, the newly encoded video looks good, so I'm assuming (perhaps incorrectly?) that the decoded video is in a proper format.  Any suggested program that can open this "raw" RGB data format so I can at least verify the decoding is correct and it's just the JPEG encoding is wrong (which I think is the case).

0 Kudos
1 Solution
Stephen_V_1
Novice
2,420 Views
I have it working now. The problem was when the function "SaveImageJPEG" gets called, the "image" that is passed in must have the color set. So now before calling SaveImageJPEG I just added the line: image.Color(IC_RGB) and the colors are coming out correctly. Also, I found a bug in the sample code which ends up not causing a problem, but is wrong. The enumeration values for "JCOLOR" are not 1-for-1 matches for the enumeration values for "IM_COLOR". In the "SaveImageJPEG" function, for each different JPEG encoding type it has the line: je_params.SetColor((JCOLOR)param.color); This is incorrect to cast param.color (an IC_IMAGE) to JCOLOR as those enumerations do not line up. As the code is implemented now, it doesn't actually end up causing an issue, but I think there should be a conversion done to the right enum before making this function call.

View solution in original post

0 Kudos
3 Replies
Ying_H_Intel
Employee
2,420 Views
Hi Stephen, first let's verify if the decode frames are correct. Have you tried to use simple_player.exe or third party player to see the decoded image? by default they are YUV420 format. You mentioned you get "raw" RGB , but the color is wrong after RGB and jpeg. From H.2664 decoder readme , the supported output color format only --f. It seems the problem come here. as "BRG24" is not same as "RGB24". It may need to add some color conversion function like BRG24 to RGB24 , then use UIC. The UMC color conversion are done by ipp-samples\audio-video-codecs\codec\color_space_converter\umc_color_space_conversion.cpp. You may obverse it and find the solution. Best Regards, Ying
0 Kudos
Stephen_V_1
Novice
2,420 Views
I am using IPP samples 7.1.0.011, I think some of the things you reference must apply to a different version, as I was unable to find them. simple_player is only for videos, I can't view images with it. I tried playing the decoded video with it, but the player continually gave errors, I don't think it knows how to play the raw decoded video format. Where is the H.264 readme you mentioned? What are the supported output colors (your post just had --f but not colors listed)? I've been looking if there is a limitation of decode color formats, but the umc_video_dec_con example applicaiton lists rgb24 as a valid color format. Here's the output of the usage print: Color formats: gray yv12 nv12 yuy2 uyvy yuv420 yuv422 yuv444 rgb24 rgb32 bgr24 bgr32 bgr565 bgr555 bgr444 Is RGB24 color format for videos the same as the JC_RGB format used in the JPEG encoder of UIC? The file umc_color_space_conversion.cpp does not exist, but I found what I believe is the same thing you mentioned at audio-video-codecs/codec/video/common/cc/src/umc_filter_color_conversion.cpp. I put prints in the color conversion routines and verified that when I run the umc_video_dec_con function with a parameter of -f rgb (as well as my own functions) I see that the color conversion routine to convert from YUV420 to RGB does get run. That confirms that I am getting out RGB frames from the H264 decoding, so I guess the question is how come UIC doesn't seem to be able to utilize that RGB format to encode the image correctly?
0 Kudos
Stephen_V_1
Novice
2,421 Views
I have it working now. The problem was when the function "SaveImageJPEG" gets called, the "image" that is passed in must have the color set. So now before calling SaveImageJPEG I just added the line: image.Color(IC_RGB) and the colors are coming out correctly. Also, I found a bug in the sample code which ends up not causing a problem, but is wrong. The enumeration values for "JCOLOR" are not 1-for-1 matches for the enumeration values for "IM_COLOR". In the "SaveImageJPEG" function, for each different JPEG encoding type it has the line: je_params.SetColor((JCOLOR)param.color); This is incorrect to cast param.color (an IC_IMAGE) to JCOLOR as those enumerations do not line up. As the code is implemented now, it doesn't actually end up causing an issue, but I think there should be a conversion done to the right enum before making this function call.
0 Kudos
Reply