Imitating Shortest Paths in Simulation for Real-World Robot Navigation and Manipulation

Explore a groundbreaking approach to robot training in this hour-long colloquium talk by Ani Kembhavi from the Allen Institute for Artificial Intelligence (AI2). Discover how imitating shortest path planners in simulation can produce agents capable of proficient navigation, exploration, and object manipulation in the real world. Learn about the SPOC agent, which achieves impressive results without human demonstrations, reinforcement learning, depth sensors, or assumptions about the target environment. Delve into the importance of scale and diversity in training data, made possible through procedurally generated simulations like ProcTHOR and HoloDeck, and the extensive 3D asset library Objaverse. Gain insights into Phone2Proc, an innovative real-to-sim method for fine-tuning robots to specific home environments. Presented by a distinguished researcher in computer vision, natural language processing, and embodiment, this talk offers valuable insights into the future of robotics and artificial intelligence.