I have tried high bit rates where the output is essentially identical to the input except for mean intensity, so this is independent of bit rate, and by eye appears to be uniform, so it is not highlight blurring/clipping. I have tried both baseline and main profiles with no change in result. I have turned off VPP denoise, scene, detail, and procamp with VppDoNotUse with no change in result.
I am not comparing raw h.264 frames, rather I am muxing to mp4 and then extracting frames with ffmpeg, but doing this on h.264 data from a different encoder does not show the same reduction in intensity.
- Media SDK 2012 R2 (x64)
- Core i7 3770K (HD 4000)
- Windows 7 x64 SP1
- latest intel graphics driver (126.96.36.19996)
Does 1.3 hw VPP or h.264 encode drop intensity?
Thanks for sharing your findings with us.
For us to make comparable investigation could you please share with us how you calculate the intensity. I'dlike to make sure we agree on the method.
Do you seesimilar intensity delta for all frames in your raw input stream and is the delta in the same range independent of what input you are using?
Are you only observing this issue when using RGB4->NV12 VPP conversion?
Here are some stats on per-frame mean pixel value differences for a short clip. So for 327 frames I compared the original to the transcoded frame. I converted both frames to grayscale, then computed the mean ofeach image, and the subtracted (originalMean - transcodedMean):
Standard Error 0.056628696
Standard Deviation 1.024024712
Sample Variance 1.048626612
Sofor theaverage frame, its grayscale mean intensity dropped by 1.9.For one frame,its grayscale meanintensity dropped by 4.88.
A different encoder has a nearly identical range magnitude, but the range is close to centered around 0, so the mean is near 0. Its standard deviation is 0.7, so a little lower but same ballpark.
For your second question, here are stats for a different clip:
Standard Error 0.062302422
Standard Deviation 0.861036598
Sample Variance 0.741384023
I understand your last question but haven't run that test yet.
This is quite strange, I do not see the same effect on my side. I tried both the driver version you used and a more recent internal driver with the same results.
First I tested the intensity difference when using VPP conversion from RGB to NV12. For the workloads I tried the greatest intensity delta I observe is ~0.35.
Using Media SDK I also produced a few H.264 video streams with varying bitrate using different content, then decoded the streams. The calculated intensity delta is very low, the greatest average intensity delta comparing encoder inoput to decoder output was ~0.30, for most workloads the delta was much smaller.
It could be that the content you are using is especially challenging? Can you share with us the content that showcases the worst intensity delta?
Tocalculate intensity, for comparable results, I used a method similar to what you describe. YUV->RGB->grayscale->intensity. This was performed on the raw input to encoder and the raw output from decoder
This is not identical to what I'm running (because I'm running from the original, uncompressed rgb3), but should be close enough.
I tried your content but did not see much difference compared to the content I used. For encode I do not really see any intensity deviation at all. For VPP RGB->NV12 color conversion slightly greater (~0.40) than what I had seen earlier.
First, I noticed now that you refer to RGB3. Does that mean you are using MFX_FOURCC_RGB3 as the input format to VPP? Note that this Media SDK capability is deprecated and not recommended to use. If you are using this please migrate to using the RGB4 type instead.
If that's not the case my guess is that we are having different results due to the algorithm you use to convert from YUV (NVC12) to RGB color space.
I am using ffmpeg toextract frames fromthe transcoded mp4 (so that is where theresulting h.264 is decoded), and am using the same command for both quicksync and x264 transcodes, so I wouldn't think they would be different, but could be.
I'll post again if I get any more relevant results.
Here is (original - sw):
Mean -0.474762046 Standard Error 0.038104205 Median -0.466354 Mode #N/A Standard Deviation 0.689043729 Sample Variance 0.474781261 Kurtosis 2.310864995 Skewness -0.007642003 Range 5.659698 Minimum -3.123901 Maximum 2.535797 Sum -155.247189 Count 327
Sothe sw range is fairly balanced around 0, so mean is near 0, and in this case the output is a little bit brighter on average.
Here is (sw - hw):
Mean 2.294726037 Standard Error 0.024461352 Median 2.075493 Mode #N/A Standard Deviation 0.442338092 Sample Variance 0.195662987 Kurtosis -1.357276659 Skewness 0.371498899 Range 1.679047 Minimum 1.452484 Maximum 3.131531 Sum 750.375414 Count 327
Thisreconfirms the previous resultbydirectly comparingthe sw and hw outputs. The sw implementation makes the content alittle brighter and the hw implementation makes the content aboutdarker.
I think my hw and sw code paths are supplying the same params.
sorry for the delayed response. We have done some more internal investigations into this.
We found some potential issues that are being looked into but not specifically linked to the intensity skew that you reported.
A topic that came up was that different YUV-RGB color conversion algorithms may be used in Media SDK vs. the method you use?
Media SDK usesBT601 for SD video and BT709 for HD video (source height greater than 576 lines). The content you provided falls into the former category. If you use BT709 for converting the frames from ffmpeg to RGB this may introduce a skew?
Also, we compared the input vs. output from VPP and separately the input vs. output from encode and did not see much intensity deviation in either case.
You could you try the same on your side by using sample_vpp and sample_encode.
Also, if there is a way for you to provide the raw RGB3 or RGB4 input then we can take a look at that case to cover all bases.