Developing Games on Intel Graphics
If you are gaming on graphics integrated in your Intel Processor, this is the place for you! Find answers to your questions or post your issues with PC games
493 Discussions

D3D12 bug: enabling stencil causes viewport to be clamped to the render target size

Triang3l
Beginner
2,349 Views

When stencil test is enabled in the pipeline state in Direct3D 12, the right and the bottom edges of the viewport are clamped to the right and the bottom edges of the render target.

Originally, this issue was reported on GitHub for the Xenia Xbox 360 emulator, which uses a 8192x8192 viewport (the maximum size of a render target on the Xbox 360) to emulate drawing directly in screen coordinates without a viewport, that is possible on the Xbox 360, without vertex position precision loss caused by the division (it's simple exponent biasing for power of two), but render targets are generally much smaller than that size — so the clear rectangle drawn by the game becomes very small.

The viewport, though, doesn't seem to be clamped to the left and the top edges — negative TopLeftX and TopLeftY are handled as normal.

Here is a screenshot comparing a rectangle drawn with stencil testing disabled ("culled", in red — with all culling disabled by PIX) and one with stencil enabled ("visible", that passed all tests — in green). The render target is 1280x512, and the viewport is at -16, -96, with the size 1536x768 (the right edge at 1520, 672).

205504708-13edb7db-8e77-4c4b-a7f4-3ac82f586b97.png

The vertex shader generates a rectangle covering the upper-left quadrant of the viewport — that is, from -16, -96, to 752, 288). This is correct for the red rectangle, and matches the behavior on Nvidia and WARP.

However, the green rectangle, that has stencil test enabled, has its bottom-right corner at 632, 208. This corresponds to the center of a viewport between -16, -96 and 1280, 512 (the right and the bottom boundaries of the render target):

-16 + (1280 - (-16)) / 2 == 632

-96 + (512 - (-96)) / 2 == 208

I've tried changing various other state parameters: whether there is a color render target, depth test enabled/disabled, depth/stencil target format, back face culling, front face winding order, DepthClipEnable, MSAA sample count — they all seem to have no effect. However, DepthStencilState.StencilEnable = TRUE appears to be enough to trigger this bug, regardless of the masks, comparison functions, and operations.

In the attachment, there's source code for a reproduction C++ console application, which runs the test on every adapter, and writes the red component of the color value for the bottom-right pixel (which should be 255 if the rectangle is large enough, 0 — the clear color — otherwise) of the intended rectangle to stderr. If launched from PIX, GPU captures will automatically be created for every adapter the test was executed on. You can experiment with the `viewport` values there.

The issue is reproducible on the Intel UHD Graphics 630 on Windows 11 64-bit with the 31.0.101.2121 driver (latest at the point of the report) and older versions throughout at least a couple of years.

The bug doesn't happen, however, on the Intel HD Graphics 4400 with the 15.40.42.5063 driver (the last driver supporting Direct3D 12 there) and older drivers for it.

I haven't verified this behavior on other GPU APIs though, but it may be worth checking them too (especially D3D11 possibly).

0 Kudos
4 Replies
karen_intel
Moderator
2,254 Views

Hey @Triang3l ty for reporting this issue. We will be doing some experiments and getting back to you in case I have questions

Stay tuned!

 

Karen

0 Kudos
Arturo_Intel
Moderator
2,060 Views

Hello @Triang3l 

I was able to replicate the issue on the UHD 630, but not on the UHD 770 or ARC 770

When viewing the results of the program running on the console, all the devices returned a 255 color value (except for the UHD 630) which was expected, which indicates to me that the problem is tied to that specific version of the driver for the UHD 630 (101.2121) as you mentioned before.

A770_UHD770.jpeg

The only thing I couldn't replicate, was to make PIX show the comparison of the rectangle drawn with the stencil disabled (red) and the other with the stencil enabled (green) on the same screen. All GPU captures taken showed only a blank rectangle.
If you can share a guide, step by step on how to show both rectangles, I would really appreciate it.

Something important to mention is the newer processor generations (11th and up) are the common target for fixes. Older products are limited to legacy updates (critical fixes and security vulnerabilities) and have a quarter-year release cadence.

Either way, I'm going to be sure to communicate the impact of this issue and hopefully hear back soon.

0 Kudos
Triang3l
Beginner
1,400 Views

Hello Arturo and Karen! Oh oops, somewhat forgot about this topic initially, but got notified about it by a user of the emulator, sorry. Thank you very much for investigating this!

 

The green and red rectangles can be displayed by selecting "Visualization: Wireframe" in the render target viewer in PIX. When it's active, PIX replaces the pixel shader and repeats the application's draw twice. The red part is drawn with depth/stencil testing force-disabled by PIX, and after it the green one is drawn on top of it respecting the application's depth/stencil state to visualize the areas where the test has passed.

 

Normally, both the red and the green draws should cover the same pixels. However, here, while the red draw with depth/stencil forced off is performed with the correct viewport, for the green one with stencil testing enabled the driver has clamped the viewport dimensions, so the object became smaller in framebuffer coordinates.

0 Kudos
karen_intel
Moderator
1,719 Views

Hey @Triang3l any update on Arturo's request?

If you can share a guide, step by step on how to show both rectangles, we would really appreciate it.

TY

Karen

0 Kudos
Reply