Explore a comprehensive explanation of the SayCan system, which combines large language models with robotic affordances to execute complex tasks. Learn how this innovative approach bridges the gap between semantic knowledge and real-world action planning. Discover the process of integrating language models with low-level skills to create a system capable of generating and executing long-horizon action sequences. Gain insights into the experimental setup, data collection methods, and the strengths and weaknesses of the SayCan system. Understand how this technology enables robots to interpret and act upon high-level, temporally extended instructions expressed in natural language.
Do As I Can, Not As I Say - Grounding Language in Robotic Affordances
Overview
Syllabus
- Introduction & Overview
- Sponsor: Zeta Alpha
- Using language models for action planning
- Combining LLMs with learned atomic skills
- The full SayCan system
- Experimental setup and data collection
- Some weaknesses & strengths of the system
- Experimental results
Taught by
Yannic Kilcher
