Functional Tokens for On-device Multimodal Models - Nexa AI

Explore the innovative approach to tokenization and on-device AI models presented in this hour-long conference talk by Alex Chen and Zach Li from USC Information Sciences Institute. Delve into the concept of functional tokens, a novel training methodology designed to overcome challenges in function calling tasks for large language models. Discover the Octopus-series models, which utilize functional tokens to achieve GPT4-level function calling accuracy with significantly reduced parameter size. Learn about the impressive performance improvements of Octopus-V2, including faster inference speed and enhanced energy efficiency compared to existing solutions. Examine the evolution of the Octopus model series, from the multimodal and multilingual capabilities of Octopus-V3 to the graph network structure of Octopus-V4. Gain insights into the industrial collaborations and recognition garnered by Nexa AI's Octopus models, including their ranking on HuggingFace and mentions by the Google Gemma team. Understand the potential applications of these models in cloud and edge collaboration, and their implications for the future of on-device AI.