Perception Contracts for Safety of ML-Enabled Systems

Explore a groundbreaking 20-minute conference talk from OOPSLA2 2023 introducing the concept of perception contracts for ensuring safety in systems using neural perception. Delve into a novel approach for reasoning about controllers interacting with environments through machine learning. Learn how perception contracts capture errors in ground-truth estimations while preserving system invariants. Discover the theory behind perception contracts and the symbolic learning algorithms developed for synthesizing them from image sets. Examine the implementation and evaluation of this approach on two real-world vision-based control systems: a lane tracking system for electric vehicles and an agricultural robot for crop row following. Gain insights into the effectiveness of synthesizing perception contracts and their ability to generalize during runtime monitoring of systems.