OpenGL GL_TEXTURE_RECTANGLE pixel shader problem

[Steve_Browne]

I'm having an issue with a shader that I wrote to convert from UYVY to RGB. This issue only seems to happen on Intel cards Linux, Mac, and Windows. Any Intel card that supports rectangular textures seems to be able to reproduce it, but if you need driver specifics I'm using Intel HD Graphics driver from 1/10/2012 (8.15.10.2622) on my Windows box. The same code works fine for NVIDIA and ATI cards.

What happens is that the resulting image looks like the Cb/Cr are reversed and there are vertical lines in the picture. See the attached image. My assumption is that the way GL_TEXTURE_RECTANGLE support is implemented is resulting in rounding errors. The reason I suggest that is because I tried to write the shader without using rectangular textures and the same vertical lines would appear as a result of rounding errors. I didn't see the Cb/Cr reversed, but that could also happen as a result of rounding errors.

I'm creating the texture with the following glTexImage2D call:
glTexImage2D(GL_TEXTURE_RECTANGLE, 0, 2, width, height, 0, GL_LUMINANCE_ALPHA, GL_UNSIGNED_BYTE, NULL);

Here is the shader code I'm using:
[cpp]struct pixel_in { int2 texcoord : TEXCOORD0; samplerRECT texture : TEXUNIT0; float4 color : COLOR0; }; struct pixel_out { float4 color : COLOR0; }; pixel_out main (pixel_in IN) { pixel_out OUT; float3 yuv; bool even = (IN.texcoord.x % 2) == 0; // Determine our neighbor to the right for even pixels and left for odd pixels int2 neighbor = {IN.texcoord.x, IN.texcoord.y}; neighbor.x += (even ? 1 : -1); // Calculations taken from http://www.fourcc.org/source/YUV420P-OpenGL-GLSLang.c float4 uy = texRECT(IN.texture, (even ? IN.texcoord : neighbor)); float4 vy = texRECT(IN.texture, (even ? neighbor : IN.texcoord)); yuv.r = 1.1643 * (even ? uy.a : vy.a) - 0.0625; yuv.g = uy.r - 0.5; yuv.b = vy.r - 0.5; OUT.color.r = yuv.r + 1.5958 * yuv.b; OUT.color.g = yuv.r - 0.39173 * yuv.g - 0.81290 * yuv.b; OUT.color.b = yuv.r + 2.017 * yuv.g; OUT.color.a = IN.color.a; return OUT; } [/cpp]

[Steve_Browne]

After some more testing I have a few more details that may be helpful. What I've noticed is that if I force the application to use power of 2 textures the problems almost all go away. However, if the power of 2 texture is not square then it results in vertically stretched video as shown in the attached screenshots (most likely because the width is double the height). If the textures are square power of 2 textures then the resulting video is fine and there is no stretching.

[Deepak_V_Intel]

Hi Steve,
Thanks for the note. We are looking at the issue and will get back to you. WOuld you be able to share the application for debugging purposes, if needed?
Thanks
-deepak

[Deepak_V_Intel]

Could you also post the specific HD graphis part that seems to be causing the issue?
Thanks
-deepak

[Steve_Browne]

I can certainly provide you with the application and any more details you need to reproduce this. As far as the specific HD graphics part causing this the laptop I have in front of me here has the issue and its a Latitude E6520 with an Intel HD Graphics 3000, driver version 8.15.10.2266. As I noted in the original post though, this problem occurs on more than just one PC. I've also reproduced it with a desktop PC that also had an Intel HD Graphics 3000 chipset in both Windows and Ubuntu 11.04. This has also been seen on an iMac that had an Intel HD Graphics chipset. I'm pretty sure this issue should be reproducible on any Intel HD Graphics chipset card.