NVIDIA swaps codecs for AI to improve video call compression

NVIDIA is teasing new video compression tech invented by its Computer Vision engineers that it says can reduce video streaming bandwidth consumption by “orders of magnitude.”

The company said AI Video Compression is designed to provide lower bitrates than current codecs like h.264 by using a neural network to avoid needing to send a stream of images. Instead, the company’s tech takes a reference image transmitted by the sender, combines it with location data on facial points like the eyes and mouth, then has a generative adversarial network reconstruct the video playback using the reference image and facial data points.

“As a result, much less data is sent over the network. Slow internet connection and limited bandwidth will not impact the quality of video meetings anymore,” the company said in a video.

RELATED: Disney unveils AI-powered video compression research

Essentially, users on the other end of a video call will see an AI-assisted recreation of a person rather than seeing actual video of the person speaking. NVIDIA said the tech can also be used for facial alignment (for video callers who look away from the camera) and for character animations.

NVIDIA is not alone in leaning into AI to help with improved video compression. Earlier this year, Disney Research and the University of California, Irvine revealed a new AI-enhanced video compression model that they said yielded less distortion and smaller bits-per-pixel rates than current coding-decoding algorithms.

“Ultimately, every video compression approach works on a trade-off,” said research team leader Stephan Mandt, a UCI assistant professor of computer science who began the project while employed at Disney Research. “If I’m allowing for larger file sizes, then I can have better image quality. If I want to have a short, really small file size, then I have to tolerate some errors. The hope is that our neural network-based approach does a better trade-off overall between file size and quality.”

Many people are still working from home during the pandemic, which has caused a substantial rise in video conferencing and calling. Even before the pandemic caused a surge in streaming video traffic, Cisco expected video to dominate IP networks. In 2018, the company estimated video traffic will quadruple by 2022 and that, at that point, video will account for 82% of all IP traffic.