By Henry C Lin and Andrew Burnes on September 20, 2022 | Featured Stories GeForce RTX GPUs NVIDIA DLSS NVIDIA RTX Ray Tracing
NVIDIA DLSS revolutionized graphics by using AI super resolution and Tensor Cores on GeForce RTX GPUs to boost frame rates while delivering crisp, high quality images that rival native resolution. Since the release of DLSS, 216 games and apps have incorporated the technology, providing faster frame rates and the performance headroom to make real-time videogame ray tracing a reality.
Today, we are excited to announce NVIDIA DLSS 3, the next revolution in neural graphics. Combining DLSS Super Resolution, all-new DLSS Frame Generation, and NVIDIA Reflex, running on the new hardware capabilities of GeForce RTX 40 Series GPUs, DLSS 3 multiplies performance by up to 4X over brute-force rendering. DLSS 3 is already being rapidly adopted by the ecosystem, with over 35 games and applications integrating the technology, the first of which launch in October.
The Evolution of NVIDIA DLSS
When we first introduced NVIDIA DLSS, we set out to redefine real-time rendering through AI-based super resolution - rendering fewer pixels and then using AI to construct sharp, higher resolution images. Less than two years later, we introduced the world to NVIDIA DLSS 2, which further improved image quality and performance with a generalized neural network that could adapt to all games and scenes without specific training. DLSS 2 is now widely adopted in 216 games and apps, along with support in Unity and Unreal Engine. DLSS 2 continues to improve through ongoing training on NVIDIA’s AI supercomputer, with 4 major updates released to date, which have brought further enhancements to image quality.
Introducing NVIDIA DLSS 3 - A Revolution in Neural Graphics
DLSS 3 is a revolutionary breakthrough in AI-powered graphics that massively boosts performance, while maintaining great image quality and responsiveness. Building upon DLSS Super Resolution, DLSS 3 adds Optical Multi Frame Generation to generate entirely new frames, and integrates NVIDIA Reflex low latency technology for optimal responsiveness. DLSS 3 is powered by the new fourth-generation Tensor Cores and Optical Flow Accelerator of the NVIDIA Ada Lovelace architecture, which powers GeForce RTX 40 Series graphics cards.
The DLSS Frame Generation convolutional autoencoder takes 4 inputs – current and prior game frames, an optical flow field generated by Ada’s Optical Flow Accelerator, and game engine data such as motion vectors and depth.
Ada’s Optical Flow Accelerator analyzes two sequential in-game frames and calculates an optical flow field. The optical flow field captures the direction and speed at which pixels are moving from frame 1 to frame 2. The Optical Flow Accelerator is able to capture pixel-level information such as particles, reflections, shadows, and lighting, which are not included in game engine motion vector calculations. In the motorcycle example below, the motion flow of the motorcyclist accurately represents that the shadow stays in roughly the same place on the screen with respect to their bike.
Whereas the Optical Flow Accelerator accurately tracks pixel level effects such as reflections, DLSS 3 also uses game engine motion vectors to precisely track the movement of geometry in the scene. In the example below, game motion vectors accurately track the movement of the road moving past the motorcyclist, but not their shadow. Generating frames using engine motion vectors alone would result in visual anomalies like stuttering on the shadow.
For each pixel, the DLSS Frame Generation AI network decides how to use information from the game motion vectors, the optical flow field, and the sequential game frames to create intermediate frames. By using both engine motion vectors and optical flow to track motion, the DLSS Frame Generation network is able to accurately reconstruct both geometry and effects, as seen in the picture below.
With DLSS 3 enabled, AI is reconstructing three-fourths of the first frame with DLSS Super Resolution, and reconstructing the entire second frame using DLSS Frame Generation. In total, DLSS 3 reconstructs seven-eighths of the total displayed pixels, increasing performance significantly!
DLSS 3 also incorporates NVIDIA Reflex, which synchronizes the GPU and CPU, ensuring optimum responsiveness and low system latency. Lower system latency makes game controls more responsive, and ensures on-screen actions occur almost instantaneously once you click your mouse or other control input. When compared to native, DLSS 3 can reduce latency by up to 2X.
In a pre-release version of Cyberpunk 2077’s new Ray Tracing: Overdrive mode, DLSS 3 enables DLSS Super Resolution, DLSS Frame Generation, and NVIDIA Reflex to boost performance by up to 4X, and increase responsiveness by 2X, compared to native resolution, all while maintaining great image quality.
NVIDIA DLSS 3 Can Double CPU Bound Performance
DLSS Frame Generation executes as a post-process on the GPU, allowing the AI network to boost frame rates even when the game is bottlenecked by the CPU. For CPU-limited games, such as those that are physics-heavy or involve large worlds, DLSS 3 allows GeForce RTX 40 Series graphics cards to render at up to twice the frame rate over what the CPU is able to compute. In Microsoft Flight Simulator for example, with the 1:1 real-world recreation of our planet, DLSS 3 boosts FPS by up to 2X.
NVIDIA DLSS 3 Multiplies Performance By Up To 4X
Across a set of games and engines, DLSS 3 helps increase GeForce RTX 40 Series performance by up to 4X compared to traditional rendering:
DLSS 3 delivers total gaming performance, advanced AI networks and Reflex software algorithms, dedicated Tensor Core and Optical Flow hardware, and an NVIDIA Supercomputer that continuously trains and improves AI networks. GeForce RTX 40 Series users get faster frame rates, quick responsiveness, and great image quality, which is only possible through full stack innovation.
NVIDIA DLSS 3: Available This October
The combination of ray tracing and AI technologies has revolutionized video games by simultaneously delivering dramatic improvements in image quality, along with massive uplifts in performance, a feat that was unheard of before the invention of RTX and DLSS.
DLSS 3 builds upon our pioneering work in AI for games, boosting performance by up to 4X, making full ray-traced games a reality, and giving gamers the power to enjoy fully simulated worlds, as demonstrated by NVIDIA Racer RTX.
And DLSS 3 has already received support from many of the world’s leading game developers and engines, with more than 35 games and applications announcing support, the first of which launch in October.
DLSS 3 games are backwards compatible with DLSS 2 technology. DLSS 3 technology is supported on GeForce RTX 40 Series GPUs. It includes 3 features: our new Frame Generation tech, Super Resolution (the key innovation of DLSS 2), and Reflex. Developers simply integrate DLSS 3, and DLSS 2 is supported by default. NVIDIA continues to improve DLSS 2 by researching and training the AI for DLSS Super Resolution, and will provide model updates for all GeForce RTX gamers, as we’ve been doing since the initial release of DLSS.
Together with the other marvels of the NVIDIA Ada Lovelace architecture, DLSS 3 delivers next-gen performance. Get further details about GeForce RTX 40 Series graphics cards, new Ada technologies, and games incorporating them on GeForce.com.
As a seasoned expert in the field of graphics processing and artificial intelligence, I can confidently affirm my knowledge in the cutting-edge technologies mentioned in the article authored by Henry C Lin and Andrew Burnes on September 20, 2022. The focus of the article revolves around NVIDIA's DLSS (Deep Learning Super Sampling) technology, specifically the introduction of DLSS 3 and its revolutionary impact on AI-powered graphics.
NVIDIA DLSS, an acronym for Deep Learning Super Sampling, is a groundbreaking technology that leverages AI super resolution and Tensor Cores on GeForce RTX GPUs to enhance frame rates while delivering high-quality images comparable to native resolution. The article highlights that since the inception of DLSS, 216 games and applications have incorporated this technology, showcasing its widespread adoption in the gaming industry.
The evolution of DLSS is traced from its initial introduction to the latest iteration, DLSS 3. DLSS 2, introduced less than two years after the first version, demonstrated significant improvements in image quality and performance through a generalized neural network adaptable to various games and scenes without specific training. The wide adoption of DLSS 2, with support in Unity and Unreal Engine and ongoing enhancements through training on NVIDIA's AI supercomputer, sets the stage for the unveiling of DLSS 3.
DLSS 3 represents a major breakthrough in AI-powered graphics by combining DLSS Super Resolution with all-new DLSS Frame Generation and NVIDIA Reflex technologies. It operates on the new hardware capabilities of the GeForce RTX 40 Series GPUs, featuring fourth-generation Tensor Cores and the Optical Flow Accelerator of the NVIDIA Ada Lovelace architecture.
The DLSS Frame Generation process is explained in detail, involving inputs from current and prior game frames, an optical flow field generated by Ada's Optical Flow Accelerator, and game engine data such as motion vectors and depth. The Optical Flow Accelerator analyzes sequential in-game frames to calculate an optical flow field, capturing pixel-level information such as particles, reflections, shadows, and lighting. This detailed analysis ensures accurate reconstruction of geometry and effects, addressing potential visual anomalies.
DLSS 3 is not only about graphics enhancement but also incorporates NVIDIA Reflex for optimal responsiveness and low system latency. The integration of DLSS 3 with NVIDIA Reflex is highlighted to reduce latency by up to 2X compared to native resolution, emphasizing the significance of real-time responsiveness in gaming.
The article provides examples of DLSS 3 in action, particularly in a pre-release version of Cyberpunk 2077's Ray Tracing: Overdrive mode, showcasing performance boosts up to 4X and increased responsiveness by 2X compared to native resolution.
Additionally, DLSS 3 is noted for its ability to double CPU-bound performance in certain scenarios, enabling GeForce RTX 40 Series graphics cards to render at up to twice the frame rate compared to CPU limitations. This is exemplified with Microsoft Flight Simulator, where DLSS 3 boosts FPS by up to 2X in CPU-limited situations.
The article concludes by emphasizing that DLSS 3 contributes to a significant increase in GeForce RTX 40 Series performance across various games and engines, achieving up to 4X improvement compared to traditional rendering methods. The backward compatibility of DLSS 3 games with DLSS 2 technology is mentioned, highlighting the seamless integration and support for both versions.
In summary, NVIDIA DLSS 3 stands as a testament to the continuous innovation in AI-powered graphics, delivering enhanced performance, image quality, and responsiveness through the integration of advanced technologies such as DLSS Super Resolution, DLSS Frame Generation, and NVIDIA Reflex on the latest hardware architecture. The article provides comprehensive insights into the intricacies of DLSS 3, making it evident that NVIDIA remains at the forefront of revolutionizing the gaming experience through AI-driven graphics technologies.
