NVIDIA has shed light on the possibility of its Ampere-based RTX 3000 GPU series being able to take advantage of Frame Generation.
During a recent interview with Digital Foundry, Bryan Catanzaro, NVIDIA Vice President of Applied Deep Learning Research, was asked about the possibility of Frame Generation running on the RTX 3000 GPU series, now that it’s running entirely on tensor cores. In his response, Catanzaro discussed the technical challenges involved, calling it primarily a question of optimization and engineering. He said that NVIDIA is launching the best Multi Frame Generation technology with the RTX 5000 Series, and that the company will see what it is able to squeeze out of older hardware in the future.
Catanzaro explained that when NVIDIA built DLSS 3 Frame Generation, it needed hardware acceleration to compute Optical Flow. There weren’t enough Tensor Cores and the GPU maker didn’t have an Optical Flow algorithm that was good enough, and it hadn’t developed a real-time Optical Flow algorithm that ran on Tensor Cores and could fit its compute budget. NVIDIA had the Optical Flow accelerator, which it had been building for years as an evolution of its video encoder technology, and it’s also been a part of the company’s automotive computer vision acceleration for self-driving cars.
According to the Vice President of Applied Deep Learning Research, it made sense for NVIDIA to use the Optical Flow accelerator for DLSS 3 Frame Generation. However, he said that the difficult part about any sort of hardware implementation of an algorithm like Optical Flow is that it’s really difficult to improve upon it. He added NVIDIA was unable to undo the failures that arose from the hardware Optical Flow with a smarter neural network until it decided to go with a fully AI-based solution. That is what him and his Applied Deep Learning Research team at NVIDIA have done for Frame Generation in DLSS 4.
