By Henry C Lin and Andrew Burnes on September 20, 2022 | Featured Stories GeForce RTX GPUs NVIDIA DLSS NVIDIA RTX Ray Tracing
NVIDIA DLSS revolutionized graphics by using AI super resolution and Tensor Cores on GeForce RTX GPUs to boost frame rates while delivering crisp, high quality images that rival native resolution. Since the release of DLSS, 216 games and apps have incorporated the technology, providing faster frame rates and the performance headroom to make real-time videogame ray tracing a reality.
Today, we are excited to announce NVIDIA DLSS 3, the next revolution in neural graphics. Combining DLSS Super Resolution, all-new DLSS Frame Generation, and NVIDIA Reflex, running on the new hardware capabilities of GeForce RTX 40 Series GPUs, DLSS 3 multiplies performance by up to 4X over brute-force rendering. DLSS 3 is already being rapidly adopted by the ecosystem, with over 35 games and applications integrating the technology, the first of which launch in October.
The Evolution of NVIDIA DLSS
When we first introduced NVIDIA DLSS, we set out to redefine real-time rendering through AI-based super resolution - rendering fewer pixels and then using AI to construct sharp, higher resolution images. Less than two years later, we introduced the world to NVIDIA DLSS 2, which further improved image quality and performance with a generalized neural network that could adapt to all games and scenes without specific training. DLSS 2 is now widely adopted in 216 games and apps, along with support in Unity and Unreal Engine. DLSS 2 continues to improve through ongoing training on NVIDIA’s AI supercomputer, with 4 major updates released to date, which have brought further enhancements to image quality.
Introducing NVIDIA DLSS 3 - A Revolution in Neural Graphics
DLSS 3 is a revolutionary breakthrough in AI-powered graphics that massively boosts performance, while maintaining great image quality and responsiveness. Building upon DLSS Super Resolution, DLSS 3 adds Optical Multi Frame Generation to generate entirely new frames, and integrates NVIDIA Reflex low latency technology for optimal responsiveness. DLSS 3 is powered by the new fourth-generation Tensor Cores and Optical Flow Accelerator of the NVIDIA Ada Lovelace architecture, which powers GeForce RTX 40 Series graphics cards.
The DLSS Frame Generation convolutional autoencoder takes 4 inputs – current and prior game frames, an optical flow field generated by Ada’s Optical Flow Accelerator, and game engine data such as motion vectors and depth.
Ada’s Optical Flow Accelerator analyzes two sequential in-game frames and calculates an optical flow field. The optical flow field captures the direction and speed at which pixels are moving from frame 1 to frame 2. The Optical Flow Accelerator is able to capture pixel-level information such as particles, reflections, shadows, and lighting, which are not included in game engine motion vector calculations. In the motorcycle example below, the motion flow of the motorcyclist accurately represents that the shadow stays in roughly the same place on the screen with respect to their bike.
Whereas the Optical Flow Accelerator accurately tracks pixel level effects such as reflections, DLSS 3 also uses game engine motion vectors to precisely track the movement of geometry in the scene. In the example below, game motion vectors accurately track the movement of the road moving past the motorcyclist, but not their shadow. Generating frames using engine motion vectors alone would result in visual anomalies like stuttering on the shadow.
For each pixel, the DLSS Frame Generation AI network decides how to use information from the game motion vectors, the optical flow field, and the sequential game frames to create intermediate frames. By using both engine motion vectors and optical flow to track motion, the DLSS Frame Generation network is able to accurately reconstruct both geometry and effects, as seen in the picture below.
With DLSS 3 enabled, AI is reconstructing three-fourths of the first frame with DLSS Super Resolution, and reconstructing the entire second frame using DLSS Frame Generation. In total, DLSS 3 reconstructs seven-eighths of the total displayed pixels, increasing performance significantly!
DLSS 3 also incorporates NVIDIA Reflex, which synchronizes the GPU and CPU, ensuring optimum responsiveness and low system latency. Lower system latency makes game controls more responsive, and ensures on-screen actions occur almost instantaneously once you click your mouse or other control input. When compared to native, DLSS 3 can reduce latency by up to 2X.
In a pre-release version of Cyberpunk 2077’s new Ray Tracing: Overdrive mode, DLSS 3 enables DLSS Super Resolution, DLSS Frame Generation, and NVIDIA Reflex to boost performance by up to 4X, and increase responsiveness by 2X, compared to native resolution, all while maintaining great image quality.
NVIDIA DLSS 3 Can Double CPU Bound Performance
DLSS Frame Generation executes as a post-process on the GPU, allowing the AI network to boost frame rates even when the game is bottlenecked by the CPU. For CPU-limited games, such as those that are physics-heavy or involve large worlds, DLSS 3 allows GeForce RTX 40 Series graphics cards to render at up to twice the frame rate over what the CPU is able to compute. In Microsoft Flight Simulator for example, with the 1:1 real-world recreation of our planet, DLSS 3 boosts FPS by up to 2X.
NVIDIA DLSS 3 Multiplies Performance By Up To 4X
Across a set of games and engines, DLSS 3 helps increase GeForce RTX 40 Series performance by up to 4X compared to traditional rendering:
DLSS 3 delivers total gaming performance, advanced AI networks and Reflex software algorithms, dedicated Tensor Core and Optical Flow hardware, and an NVIDIA Supercomputer that continuously trains and improves AI networks. GeForce RTX 40 Series users get faster frame rates, quick responsiveness, and great image quality, which is only possible through full stack innovation.
NVIDIA DLSS 3: Available This October
The combination of ray tracing and AI technologies has revolutionized video games by simultaneously delivering dramatic improvements in image quality, along with massive uplifts in performance, a feat that was unheard of before the invention of RTX and DLSS.
DLSS 3 builds upon our pioneering work in AI for games, boosting performance by up to 4X, making full ray-traced games a reality, and giving gamers the power to enjoy fully simulated worlds, as demonstrated by NVIDIA Racer RTX.
And DLSS 3 has already received support from many of the world’s leading game developers and engines, with more than 35 games and applications announcing support, the first of which launch in October.
DLSS 3 games are backwards compatible with DLSS 2 technology. DLSS 3 technology is supported on GeForce RTX 40 Series GPUs. It includes 3 features: our new Frame Generation tech, Super Resolution (the key innovation of DLSS 2), and Reflex. Developers simply integrate DLSS 3, and DLSS 2 is supported by default. NVIDIA continues to improve DLSS 2 by researching and training the AI for DLSS Super Resolution, and will provide model updates for all GeForce RTX gamers, as we’ve been doing since the initial release of DLSS.
Together with the other marvels of the NVIDIA Ada Lovelace architecture, DLSS 3 delivers next-gen performance. Get further details about GeForce RTX 40 Series graphics cards, new Ada technologies, and games incorporating them on GeForce.com.
