The integration of artificial intelligence in game rendering is on the rise, prompting Microsoft to incorporate AI capabilities into the framework for upcoming graphics hardware.
On Thursday, the company unveiled additions to its DirectX API: DirectX Linear Algebra and the DirectX Compute Graph Compiler. Previews of these features are scheduled for release later this year.
Microsoft's official description of these tools clarifies their intended roles in the ecosystem.
In a recent blog entry, Adele Parsons, Microsoft's graphics program manager, emphasized that machine learning has transcended its role as a mere enhancement or after-effect. She noted its growing presence across the entire graphics process, from frame creation and content development to fulfilling developers' creative goals. DirectX is adapting to accommodate this shift, treating ML as an essential component equivalent to conventional rendering tasks.
Many tech-savvy users are familiar with how AI and machine learning enhance graphics processing units. For instance, upscaling involves the GPU generating images at a simpler, reduced resolution before applying AI to enhance them to the target clarity. Frame generation renders select frames and employs AI to estimate the visuals for the frames in between, which may add slight delays but significantly boosts frame rates for smoother gameplay.
These methods enhance the GPU's core output, enabling lower-end or integrated graphics solutions, such as Intel's forthcoming Panther Lake, to perform comparably to previous standalone GPUs in gaming scenarios.
During his presentation at the Game Developers Conference in San Francisco, Microsoft software engineering manager Max McMullen shared the stage with representatives from AMD, Intel, and Nvidia to demonstrate their endorsement of the initiative.
DirectX Linear Algebra primarily facilitates the mathematical operations required for AI. Conventional GPUs rely on vector-matrix computations for 3D modeling and illumination effects. In contrast, AI-optimized processors, including specialized workstation GPUs, leverage matrix-matrix operations. Components like Nvidia's Tensor cores, which have gained prominence, execute these matrix-matrix tasks.
Rather than granting developers direct oversight of AI functions, DirectX Linear Algebra addresses a key insight from Microsoft: techniques such as temporal upscaling rely on matrix mathematics, which integrate effectively with shaders. These shaders serve as GPU rendering directives, often pre-loaded prior to gameplay—a practice Microsoft aims to minimize.
The DirectX Compute Graph Compiler holds greater promise for innovation. Earlier solutions, like the initial version of AMD's FSR, analyzed frame differences on a pixel-by-pixel basis. Contemporary iterations of FidelityFX Super Resolution and Nvidia's DLSS now incorporate comprehensive scene analysis. Rather than adjusting individual pixels, the AI determines optimal pixel positions and applies them directly.
This approach ensures that complex interactions, such as an object passing behind another, are accurately captured, unlike pixel-based methods. Microsoft's goal is to embed this advanced processing directly within the DirectX system.
Utilizing both DirectX features, a game could dynamically communicate with the GPU to build custom shaders, extending compatibility to hardware released long after the game's launch, according to Nvidia distinguished engineer Don Brittain.
Certain players oppose AI-generated 'synthetic' frames that approximate native rendering. These DirectX advancements would amplify such techniques. Discussions included neural texture compression, where AI reconstructs details from compressed data, and neural lighting, which simulates light paths intelligently.
The benefits involve expanding access to sophisticated effects for more users. According to McMullen, neural texture compression could cut memory and storage demands for textures by as much as 30 percent. Neural radiance fields might lessen reliance on specialized ray-tracing hardware, broadening photorealistic path-tracing capabilities.
These developments remain in early stages. Microsoft plans a private preview of the DirectX Compute Graph Compiler this summer, with DirectX Linear Algebra entering public preview in April. Full incorporation into DirectX and widespread adoption will follow thereafter.
Mark has contributed to PCWorld for the past ten years, drawing on three decades of tech reporting expertise. He has produced more than 3,500 pieces for PCWorld, focusing on processors, hardware accessories, and Windows systems, among others. His work has appeared in PC Magazine, Byte, eWEEK, Popular Science, and Electronic Buyers' News, earning a Jesse H. Neal Award for news coverage. Recently, he cleared out numerous Thunderbolt docks and USB-C hubs from his workspace due to space constraints.