Bug in TransformQuantLumaDC_H264?

[mkalman]

I'm using the ippi TransformQuantLumaDC_H264 call in the UMC H.264 encoder sample with the IPP 6.0.0.062 libraries. I'm statically linking with the merged version of the libraries. I'm running on an Intel Xeon 3.19 GHz processor. When I use the call to transfrom the input by setting "NeedTransform" argument to 1, the transformed coefficients written to the "pTBlock" array are usually correct. However, if the DC component of the array being transformed is high, the DC coefficient (in position 0 of the array written to pTBlock by the call) is incorrect. It is typically clipped to the value 16383. For instance if the input array being transformed is:

pSrcDst --> [-208 +1197 +2895 +3296 +1191 +3296 +3296 +3296 +2796
+3296 +3296 +3296 +3293 +3296 +3296 +3296]

the output returned to the pTBlock array (and the coefficients that are quantized and returned to pSrcDst) is

pTBlock --> [+16383 -3905 -1806 -2207 -3803 -3402 -1303 -1704 -1701 -1300 +799 +398 -2198 -1797 +302 -99]

However, if I compute the transform as specified in the standard the transform coefficients should be

pTBlock --> [+22318 -3905 -1806 -2207 -3803 -3402 -1303 -1704 -1701 -1300
+799 +398 -2198 -1797 +302 -99]


With some input sequences the error causes very noticeable encoding artifacts where the DC value of an entire 16x16 block is wrong. Is this a known issue?

[Ying_H_Intel]

Hello mkalman

I am trying to build a test code here for this problem. But I get some difficulties to verify the problem. Bellow is one code I test for the DCT transforms. The output seemsneither the value 16383,nor 22318.

Could youprovide some more information about this problem?
1. What is the QP number that will produce the error result?
2. What is the result if you run the below code on your machine?

Regards,
Ying


#include
#include

void main()
{

ippStaticInit();

// Print the version of ipp being used
const IppLibraryVersion* lib = ippvcGetLibVersion();
printf("%s %s %d.%d.%d.%dn", lib->Name, lib->Version,lib->major, lib->minor, lib->majorBuild, lib->build);


short inputdata[4*4] ={-208, 1197, 2895, 3296, 1191, 3296, 3296, 3296,2796,
3296, 3296, 3296, 3293, 3296, 3296, 3296};

Ipp16s pScanMatrix[16]={ 0,1,5,6,2,4,7,12,3,8,11,13,9,10,14,15};
int i;
Ipp32s QP=11;
Ipp8s NumLevels;
Ipp8u LastCoeff;
Ipp16s pTBlock[16];

ippiTransformQuantLumaDC_H264_16s_C1I( inputdata,pTBlock, QP, &NumLevels, 1, pScanMatrix,
&LastCoeff);

for(i=0;i<16;i++)
{
printf("%dt", pTBlock);

}

return ;
}

I try it on several machines, the results aresimiliar. please seebelow
ippvcv8l.lib 6.0 build 167.26 6.0.167.632
22062 -3905 -1806 -2207 -3803 -3402 -1303 -1704 -1701 -1300 799 398 -2198 -1797 302 -99
Press any key to continue . . .