Nvidia DLSS 5 Works Off 2D Frames and Motion Vectors, Artists Seemingly Can’t Make Big Model Changes

by

Following some criticism after Nvidia’s presentation of DLSS 5, the company has released more information about the technology. Nvidia’s Jacob Freeman explained how it works and was trained in a recent conversation with YouTuber Daniel Owen, and also discussed its compatibility with physically based rendering (PBR) materials.

Owen’s questions focused on how DLSS 5 creates images: it uses existing 2D frames and motion data to generate new ones. Freeman confirmed this process. The conversation then explored whether DLSS 5 essentially ‘reimagines’ a game’s image, similar to how an AI might upscale or improve a regular screenshot.

According to Freeman, Nvidia has developed technology that can interpret the details of an image, recognizing things like textures of hair and cloth, individual characters and objects, and even how light affects the scene. Importantly, this understanding is based on analyzing just one still image.

During a discussion about how DLSS 5 impacts character appearances, Nvidia’s Owen highlighted changes to a character’s hairline in the Starfield demo and added makeup on a character in Resident Evil Requiem. Nvidia’s Freeman clarified that the core character models themselves haven’t been modified. He explained that these observations are from a very early version of the technology and that Nvidia is only confirming the underlying geometry remains unchanged, not addressing any visual alterations caused by DLSS 5.

DLSS 5 also doesn’t currently use information from artists about how materials should look. Instead, it guesses material properties by analyzing the image and movement within the game. According to Freeman, the technology determines these properties directly from what’s already rendered on screen. While Freeman previously described the tech as improving the realism of materials – specifically their roughness and how they interact with light – Owens believes this description isn’t entirely accurate.

Nvidia has stated that developers and artists can adjust three main features of DLSS 5: its strength, color, and how it affects specific areas of the image. However, it’s unclear if they’ll have more precise control – for example, to prevent details like makeup from being artificially enhanced on characters. Nvidia representative Freeman emphasized the three core controls and explained that developers can exclude certain objects or areas from the enhancement process. Essentially, beyond completely disabling DLSS 5 for a specific element – as seen with the character Grace in a Resident Evil Requiem demonstration – developers don’t appear to have options for granular adjustments like turning off makeup enhancement.

We’re still talking with developers to learn how they want to use and manage this technology,” he explained. “Our goal with DLSS 5 is to give them a tool to bring their creative ideas to life, instead of being restricted by the usual limits of real-time graphics rendering.”

Owen wondered if DLSS 5, which appears to function based on just one frame from the game, could also apply to objects that aren’t currently visible on screen. However, Freeman confirmed that the technology only uses the visuals and motion data from what you see in that single frame.

So, I’ve been following the whole DLSS 5 drama, and it’s getting pretty interesting. I just read what an animator working on Death Stranding 2: On the Beach thinks about the tech, and it’s worth checking out. Apparently, developers weren’t really clued in on how DLSS 5 worked before Nvidia showed it off, which is kinda wild. Now, Nvidia’s CEO, Jensen Huang, is pushing back hard, saying anyone criticizing it is totally off base. It’s definitely a messy situation, and I’m curious to see how it all plays out.

Read More

2026-03-20 16:11