Efficient Model Development in the Era of Large Language Models

Join a research seminar exploring efficient development methods for large language models, presented by Dr. Tu Vu, Assistant Professor at Virginia Tech and former Research Scientist at Google DeepMind. Delve into two key research questions surrounding post-training development of large language models: optimizing dataset proportions for multitask mixture fine-tuning and integrating new capabilities without compromising existing performance. Learn about the innovative Foundational Large Auto-rater Models (FLAMe) and its tail-patch fine-tuning approach for evaluating dataset impact on targeted distributions. Discover how large-scale model merging offers a promising solution for efficiently incorporating new capabilities while preserving existing performance. Gain insights from Dr. Vu's experience contributing to models like Gemini and Flan-T5, and his work on parameter-efficient transfer learning through soft prompts that has been widely adopted at Google for tuning large language models.