Building Embodied Autonomous Agents

Explore the cutting-edge developments in embodied autonomous agents through this comprehensive seminar by Ruslan Salakhutdinov at MIT. Delve into modular agent design for visual navigation, natural language instruction following, efficient exploration, and long-term planning. Learn about the innovative Self-supervised Embodied Active Learning (SEAL) framework, which utilizes 3D semantic maps to enhance both action and perception in a self-supervised manner. Discover how SEAL improves object detection and instance segmentation while boosting performance in object goal navigation tasks. Examine a novel embodied instruction following method that employs structured representations and semantic search policies to achieve state-of-the-art performance in the ALFRED environment. Gain insights into the benefits of explicit spatial memory and semantic search policies for more robust and generalizable state-tracking and guidance. Throughout the seminar, explore topics such as physical intelligence, goal-conditioned navigation, active neural SLAM, domain generalization, topological maps, and the transition from simulation to real-world applications in building intelligent agents.