This technical deep dive was written by Jonathan Dupuy, and is a joint work with Anis Benyoub, Laurent Belcour, Mariusz Merecki, and Thomas Chambon as part of their research efforts at Intel Labs.
Highlights:
- Microsoft will showcase Cooperative Vectors on DirectX during the Advanced Graphics Summit at the Game Developer Conference, which takes place in San Francisco on 17-21 March 2025.
- Anis Benyoub from Intel’s Graphics Research team will join Microsoft onstage to explain how the new DirectX feature enabled a 10x gain in inference performance for Neural Block Texture Compression.
- Intel is pleased to offer support for Cooperative Vectors in Intel® Xe GPU architectures so that players can enjoy a visually compelling gaming experience at high performance.
Microsoft recently announced that Cooperative Vectors are coming to DirectX, and they will be showcasing this feature during the Advanced Graphics Summit at the 2025 Game Developer Conference, which takes place in San Francisco from March 17th to 21st.
Cooperative Vectors allow the multiplication of matrices with arbitrarily sized vectors to be run on any shader stage. They can therefore be used not only for training AI models, e.g., using compute shaders, but also for real-time inference, and specifically per-pixel inference, to accelerate the execution of neural rendering techniques on hardware with AI acceleration. With this new feature, developers can unleash the full power of the XMX units on modern Intel GPUs. Cooperative Vector support will be available on discrete GPUs, such as Intel® Arc™ (A series and B series), and built-in Intel® Arc™ GPUs in Intel® Core™ Ultra Processors (Series 2).
Intel is excited to be joining Microsoft on stage to demonstrate the benefits of Cooperative Vectors on Intel products. Anis Benyoub from Intel’s Graphics Research team will explain how this new DirectX feature enabled a 10x gain in inference performance for Neural Block Texture Compression.
Why Neural Block Texture Compression?
Nowadays, game textures commonly reach 4K resolution and are often composed of many channels encoding surface details such as albedo, normal, roughness, metalness, ambient occlusion, etc. For example, the T-Rex asset shown in Figure 1 uses eight 4096 x 4096 PBR textures with 13 individual channels each: this totals over 200 million texture values per texture to store in video memory, and that is for a single asset!
Figure 1. T-Rex asset shown on left, and its associated textures shown on right.
Even with the help of hardware texture compression (such as Block Compression or BC), such amounts of texture data can saturate disk capacities on current-gen consoles. Many games, such as Call of Duty, have to continuously stream texture content from the internet, thereby saving over 100GB in disk space (see: “Extending In-Game Textures using CDNs for Call of Duty: MWII” by Chris Fowler, GDC 23). There is therefore a big challenge to ramp up texture compression, and it turns out that combining AI with Cooperative Vectors is a great way to reach this goal!
Neural Block Texture Compression
A fundamental limitation of traditional texture compression is that it only works on textures of up to four channels (RGBA). In contrast, Neural Block Texture Compression operates on arbitrary numbers of channels and benefits from it. This makes sense for, e.g., our T-Rex asset, because all 13 channels are correlated, meaning that they share structures and patterns across channels, as shown in Figure 2 below. These correlations can be captured and encoded within a neural network to reach up to five times the compression ratio of a conventional BC compressor.
Figure 2. Some of the texture channels of the T-Rex asset. Although each channel encodes a different surface attribute, they all share very noticeable structural similarities that a neural network can take advantage of to reach higher compression ratios.
Cooperative Vectors Acceleration
Our neural compression method is very similar to the one developed by Ubisoft (see “Real-Time Neural Materials using Block-compressed Features,” by Weinreich et al.), where texture channels are compressed to a large matrix that we use as a Multi-Layer Perceptron. Decompression thus becomes an inference that consists of a few vector/matrix multiplications. Thanks to the new DirectX Cooperative Vectors API, we can directly use AI hardware and speed up the computation by up to a factor of 10 on Intel® Arc™ (B series) GPUs.
Conclusion
Combining neural textures is a promising way to reduce texture memory requirements. With new DirectX Cooperative Vectors support, neural textures can also achieve practical real-time performance using hardware AI acceleration on modern GPUs. Intel is delighted to offer support for this feature in Intel® Xe GPU architectures so that players can enjoy a visually compelling gaming experience at high performance. For additional information on cooperative vectors at GDC, make sure to check out Microsoft’s blog post.