- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have modifiedsample codeto read from an RGB3 AVI and encode to h.264 using VPP for RGB4 to NV12 color format conversion. I'm using hw API 1.3. The output has lower mean intensity than the input (109 down to 105 for the frame I'm looking at).
I have tried high bit rates where the output is essentially identical to the input except for mean intensity, so this is independent of bit rate, and by eye appears to be uniform, so it is not highlight blurring/clipping. I have tried both baseline and main profiles with no change in result. I have turned off VPP denoise, scene, detail, and procamp with VppDoNotUse with no change in result.
I am not comparing raw h.264 frames, rather I am muxing to mp4 and then extracting frames with ffmpeg, but doing this on h.264 data from a different encoder does not show the same reduction in intensity.
My system:
- Media SDK 2012 R2 (x64)
- Core i7 3770K (HD 4000)
- Windows 7 x64 SP1
- latest intel graphics driver (8.15.10.2696)
Does 1.3 hw VPP or h.264 encode drop intensity?
Ryan
I have tried high bit rates where the output is essentially identical to the input except for mean intensity, so this is independent of bit rate, and by eye appears to be uniform, so it is not highlight blurring/clipping. I have tried both baseline and main profiles with no change in result. I have turned off VPP denoise, scene, detail, and procamp with VppDoNotUse with no change in result.
I am not comparing raw h.264 frames, rather I am muxing to mp4 and then extracting frames with ffmpeg, but doing this on h.264 data from a different encoder does not show the same reduction in intensity.
My system:
- Media SDK 2012 R2 (x64)
- Core i7 3770K (HD 4000)
- Windows 7 x64 SP1
- latest intel graphics driver (8.15.10.2696)
Does 1.3 hw VPP or h.264 encode drop intensity?
Ryan
Link Copied
9 Replies
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Ryan,
Thanks for sharing your findings with us.
For us to make comparable investigation could you please share with us how you calculate the intensity. I'dlike to make sure we agree on the method.
Do you seesimilar intensity delta for all frames in your raw input stream and is the delta in the same range independent of what input you are using?
Are you only observing this issue when using RGB4->NV12 VPP conversion?
Thanks,
Petter
Thanks for sharing your findings with us.
For us to make comparable investigation could you please share with us how you calculate the intensity. I'dlike to make sure we agree on the method.
Do you seesimilar intensity delta for all frames in your raw input stream and is the delta in the same range independent of what input you are using?
Are you only observing this issue when using RGB4->NV12 VPP conversion?
Thanks,
Petter
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
My intensity metric is mean pixel value after converting to 8u grayscale.
Here are some stats on per-frame mean pixel value differences for a short clip. So for 327 frames I compared the original to the transcoded frame. I converted both frames to grayscale, then computed the mean ofeach image, and the subtracted (originalMean - transcodedMean):
Sofor theaverage frame, its grayscale mean intensity dropped by 1.9.For one frame,its grayscale meanintensity dropped by 4.88.
A different encoder has a nearly identical range magnitude, but the range is close to centered around 0, so the mean is near 0. Its standard deviation is 0.7, so a little lower but same ballpark.
For your second question, here are stats for a different clip:
I understand your last question but haven't run that test yet.
Here are some stats on per-frame mean pixel value differences for a short clip. So for 327 frames I compared the original to the transcoded frame. I converted both frames to grayscale, then computed the mean ofeach image, and the subtracted (originalMean - transcodedMean):
Mean 1.912494914
Standard Error 0.056628696
Median 1.742355
Mode #N/A
Standard Deviation 1.024024712
Sample Variance 1.048626612
Kurtosis -0.510323895
Skewness 0.102027743
Range 5.714798
Minimum -0.827156
Maximum 4.887642
Sum 625.385837
Count 327
Sofor theaverage frame, its grayscale mean intensity dropped by 1.9.For one frame,its grayscale meanintensity dropped by 4.88.
A different encoder has a nearly identical range magnitude, but the range is close to centered around 0, so the mean is near 0. Its standard deviation is 0.7, so a little lower but same ballpark.
For your second question, here are stats for a different clip:
Mean 1.036684283
Standard Error 0.062302422
Median 1.271118
Mode #N/A
Standard Deviation 0.861036598
Sample Variance 0.741384023
Kurtosis -0.191864263
Skewness -0.262728729
Range 4.726814
Minimum -1.107979
Maximum 3.618835
Sum 198.006698
Count 191
I understand your last question but haven't run that test yet.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
For the last question, when I use IPP for the RGB to YUV colorspace conversion (to avoid using VPP), the intensity is greatly increased in the output. So IPP and Quicksync seem to have a different RGB->YUV conversion regarding intensity so this isolation is not instructive.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Ryan,
This is quite strange, I do not see the same effect on my side. I tried both the driver version you used and a more recent internal driver with the same results.
First I tested the intensity difference when using VPP conversion from RGB to NV12. For the workloads I tried the greatest intensity delta I observe is ~0.35.
Using Media SDK I also produced a few H.264 video streams with varying bitrate using different content, then decoded the streams. The calculated intensity delta is very low, the greatest average intensity delta comparing encoder inoput to decoder output was ~0.30, for most workloads the delta was much smaller.
It could be that the content you are using is especially challenging? Can you share with us the content that showcases the worst intensity delta?
Tocalculate intensity, for comparable results, I used a method similar to what you describe. YUV->RGB->grayscale->intensity. This was performed on the raw input to encoder and the raw output from decoder
Regards,
This is quite strange, I do not see the same effect on my side. I tried both the driver version you used and a more recent internal driver with the same results.
First I tested the intensity difference when using VPP conversion from RGB to NV12. For the workloads I tried the greatest intensity delta I observe is ~0.35.
Using Media SDK I also produced a few H.264 video streams with varying bitrate using different content, then decoded the streams. The calculated intensity delta is very low, the greatest average intensity delta comparing encoder inoput to decoder output was ~0.30, for most workloads the delta was much smaller.
It could be that the content you are using is especially challenging? Can you share with us the content that showcases the worst intensity delta?
Tocalculate intensity, for comparable results, I used a method similar to what you describe. YUV->RGB->grayscale->intensity. This was performed on the raw input to encoder and the raw output from decoder
Regards,
Petter
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Here is a fairly high bitrate encode using x264:
http://dl.dropbox.com/u/62496791/x264_3000_h.264
This is not identical to what I'm running (because I'm running from the original, uncompressed rgb3), but should be close enough.
http://dl.dropbox.com/u/62496791/x264_3000_h.264
This is not identical to what I'm running (because I'm running from the original, uncompressed rgb3), but should be close enough.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Ryan,
I tried your content but did not see much difference compared to the content I used. For encode I do not really see any intensity deviation at all. For VPP RGB->NV12 color conversion slightly greater (~0.40) than what I had seen earlier.
First, I noticed now that you refer to RGB3. Does that mean you are using MFX_FOURCC_RGB3 as the input format to VPP? Note that this Media SDK capability is deprecated and not recommended to use. If you are using this please migrate to using the RGB4 type instead.
If that's not the case my guess is that we are having different results due to the algorithm you use to convert from YUV (NVC12) to RGB color space.
Regards,
I tried your content but did not see much difference compared to the content I used. For encode I do not really see any intensity deviation at all. For VPP RGB->NV12 color conversion slightly greater (~0.40) than what I had seen earlier.
First, I noticed now that you refer to RGB3. Does that mean you are using MFX_FOURCC_RGB3 as the input format to VPP? Note that this Media SDK capability is deprecated and not recommended to use. If you are using this please migrate to using the RGB4 type instead.
If that's not the case my guess is that we are having different results due to the algorithm you use to convert from YUV (NVC12) to RGB color space.
Regards,
Petter
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
My input file is RGB3, I am converting to RGB4 manually during AVI read so VPP is receiving RGB4. RGB3 is apparently not supported by VPP in version 1.3 API, judging by the MFX_ERR_UNSUPPORTED code I get.
I am using ffmpeg toextract frames fromthe transcoded mp4 (so that is where theresulting h.264 is decoded), and am using the same command for both quicksync and x264 transcodes, so I wouldn't think they would be different, but could be.
I'll post again if I get any more relevant results.
Ryan
I am using ffmpeg toextract frames fromthe transcoded mp4 (so that is where theresulting h.264 is decoded), and am using the same command for both quicksync and x264 transcodes, so I wouldn't think they would be different, but could be.
I'll post again if I get any more relevant results.
Ryan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I ran the processing (both VPP and encode) using sw 1.4 and hw 1.3 implementations, and compared them.
Here is (original - sw):
Mean -0.474762046 Standard Error 0.038104205 Median -0.466354 Mode #N/A Standard Deviation 0.689043729 Sample Variance 0.474781261 Kurtosis 2.310864995 Skewness -0.007642003 Range 5.659698 Minimum -3.123901 Maximum 2.535797 Sum -155.247189 Count 327
Sothe sw range is fairly balanced around 0, so mean is near 0, and in this case the output is a little bit brighter on average.
Here is (sw - hw):
Mean 2.294726037 Standard Error 0.024461352 Median 2.075493 Mode #N/A Standard Deviation 0.442338092 Sample Variance 0.195662987 Kurtosis -1.357276659 Skewness 0.371498899 Range 1.679047 Minimum 1.452484 Maximum 3.131531 Sum 750.375414 Count 327
Thisreconfirms the previous resultbydirectly comparingthe sw and hw outputs. The sw implementation makes the content alittle brighter and the hw implementation makes the content aboutdarker.
I think my hw and sw code paths are supplying the same params.
Ryan
Here is (original - sw):
Mean -0.474762046 Standard Error 0.038104205 Median -0.466354 Mode #N/A Standard Deviation 0.689043729 Sample Variance 0.474781261 Kurtosis 2.310864995 Skewness -0.007642003 Range 5.659698 Minimum -3.123901 Maximum 2.535797 Sum -155.247189 Count 327
Sothe sw range is fairly balanced around 0, so mean is near 0, and in this case the output is a little bit brighter on average.
Here is (sw - hw):
Mean 2.294726037 Standard Error 0.024461352 Median 2.075493 Mode #N/A Standard Deviation 0.442338092 Sample Variance 0.195662987 Kurtosis -1.357276659 Skewness 0.371498899 Range 1.679047 Minimum 1.452484 Maximum 3.131531 Sum 750.375414 Count 327
Thisreconfirms the previous resultbydirectly comparingthe sw and hw outputs. The sw implementation makes the content alittle brighter and the hw implementation makes the content aboutdarker.
I think my hw and sw code paths are supplying the same params.
Ryan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Ryan,
sorry for the delayed response. We have done some more internal investigations into this.
We found some potential issues that are being looked into but not specifically linked to the intensity skew that you reported.
A topic that came up was that different YUV-RGB color conversion algorithms may be used in Media SDK vs. the method you use?
Media SDK usesBT601 for SD video and BT709 for HD video (source height greater than 576 lines). The content you provided falls into the former category. If you use BT709 for converting the frames from ffmpeg to RGB this may introduce a skew?
http://en.wikipedia.org/wiki/Rec._709
Also, we compared the input vs. output from VPP and separately the input vs. output from encode and did not see much intensity deviation in either case.
You could you try the same on your side by using sample_vpp and sample_encode.
Also, if there is a way for you to provide the raw RGB3 or RGB4 input then we can take a look at that case to cover all bases.
Regards,
sorry for the delayed response. We have done some more internal investigations into this.
We found some potential issues that are being looked into but not specifically linked to the intensity skew that you reported.
A topic that came up was that different YUV-RGB color conversion algorithms may be used in Media SDK vs. the method you use?
Media SDK usesBT601 for SD video and BT709 for HD video (source height greater than 576 lines). The content you provided falls into the former category. If you use BT709 for converting the frames from ffmpeg to RGB this may introduce a skew?
http://en.wikipedia.org/wiki/Rec._709
Also, we compared the input vs. output from VPP and separately the input vs. output from encode and did not see much intensity deviation in either case.
You could you try the same on your side by using sample_vpp and sample_encode.
Also, if there is a way for you to provide the raw RGB3 or RGB4 input then we can take a look at that case to cover all bases.
Regards,
Petter
Reply
Topic Options
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page