PlayStation has filed a patent for frame generation technology that predicts the next game frame before it is rendered.
According to the patent (via Patentlyze), PlayStation is exploring an AI-powered approach to frame generation that predicts upcoming frames using an autoencoder model. The system analyses previous frames, motion vectors that indicate the movement of objects, and recent user input to predict what the next frame should look like. The model consists of two neural networks i.e. an encoder and a decoder. The encoder converts an existing frame into a compact representation called an image embedding, which captures important visual information. The decoder then uses this information to generate a synthetic frame that can be displayed before the next real frame is ready.
This approach differs from conventional frame generation, which typically requires two rendered frames to generate an additional frame between them. Waiting for the subsequent frame can introduce extra input latency in games. Sony Interactive Entertainment’s proposed technique instead uses frame extrapolation, predicting a future frame based on information already available. By displaying the predicted frame while the graphics hardware renders the next real one, the technology may help maintain smoother gameplay without waiting for the complete rendering process.
While predicting an intermediate frame between two known images is relatively straightforward, predicting a future frame without knowing exactly what happens next presents a much greater challenge. Sudden camera movements, unexpected player actions, and newly appearing objects can all result in visual inaccuracies. The patent’s illustration demonstrate this limitation, showing differences between predicted and actual frames. Sony Interactive Entertainment also describes a simpler non-AI fallback that shifts blocks of pixels along their motion.
Earlier this year, during an interview with Digital Foundry, PlayStation architect Mark Cerny mentioned that FSR frame generation was co-developed between Sony Interactive Entertainment and AMD, and that he looked forward to discussing it more in the future. If the patent is incorporated into the technology for PlayStation consoles, it may deliver lower latency than existing techniques.
