Gemino: Practical and Robust Neural Compression for Video Conferencing

Explore a groundbreaking conference talk from NSDI '24 that introduces Gemino, an innovative neural compression system designed to enhance video conferencing quality. Delve into the challenges faced by current video codecs during poor network conditions and discover how Gemino addresses these issues through a novel high-frequency-conditional super-resolution pipeline. Learn about the system's ability to upsample low-resolution frames while enhancing high-frequency details using information from a single high-resolution reference image. Understand the multi-scale architecture that enables Gemino to scale to resolutions comparable to 720p and its personalization capabilities for improved fidelity at low bitrates. Gain insights into the implementation of Gemino using aiortc, an open-source Python implementation of WebRTC, and its real-time performance on a Titan X GPU. Discover how Gemino achieves 2.2–5x lower bitrate than traditional video codecs while maintaining perceptual quality, potentially revolutionizing video conferencing experiences.