The World Is Worth How Many APIs? A Thought Experiment

Explore a thought-provoking conference talk presented at the NL Reasoning workshop during ACL 2024, delving into the question of how many primitive actions or APIs are necessary for versatile embodied AI agents. Examine a framework that uses GPT-4 to generate Pythonic programs as agent policies, bootstrapping a universe of APIs by reusing existing ones and fabricating new ones when needed. Learn about the application of this pipeline to wikiHow tutorials, resulting in an action space of over 300 APIs necessary for capturing diverse tasks in the physical world. Discover insights from automatic and human analysis of the induction output, revealing the effectiveness of API reuse and creation. Gain perspective on the limitations of existing simulators in supporting the induced APIs, highlighting the need for more action-rich embodied environments.