Robotic Dexterous Manipulation - Advances in Learning and Teleoperation

Explore cutting-edge advancements in robotic dexterous manipulation through this insightful 58-minute talk by Luis Pineda, a Senior Research Engineer at FAIR (Meta). Delve into the challenges of building robots with general skills for handling novel objects and performing multiple tasks. Discover how combining advances in perception, planning, and hardware, along with merging classical robotics techniques and purely learned methods, can lead to more versatile and capable robotic systems. Learn about three key projects: NCF-v2, a model for learning extrinsic contact to improve policy performance on insertion tasks; NeuralFeels, a visuo-tactile SLAM system for real-time object shape and pose estimation during in-hand manipulation; and RotateIt, a reinforcement learning-based method for in-hand object rotation using vision and touch. Gain insights into the diverse work conducted at FAIR towards achieving general dexterous manipulation, and understand how the integration of perception and planning advances is crucial for enhancing system performance and capabilities in robotics.