Segmenting and Tracking Every Pixel

Explore video panoptic segmentation in this comprehensive conference talk. Dive into the challenges of assigning semantic classes and track identities to every pixel in a video. Learn about the introduction of two new benchmark datasets, KITTI-STEP and MOTChallenge-STEP, designed to provide long video sequences for studying long-term pixel-precise segmentation and tracking under real-world conditions. Discover the novel evaluation metric, Segmentation and Tracking Quality (STQ), which balances semantic and tracking aspects for sequences of arbitrary length. Examine the evolution of visual scene understanding, the importance of segmentation and tracking, and the process of dataset creation. Explore existing datasets, annotation processes, and track length distributions. Gain insights into metric design, formal definitions, and the components of STQ. Investigate extensions to tracking single-frame baselines and review results on the KITTI-STEP dataset. Access valuable resources and gain a comprehensive understanding of this cutting-edge research in computer vision and machine learning.