Explore the technical challenges and potential solutions for scaling transformer models to handle longer context in this 32-minute Toronto Machine Learning Series talk presented by Cohere's Technical Staff Member Bowen Yang. Dive into current methodologies for extending context windows, examine the obstacles faced from both modeling and framework perspectives, and discover emerging directions for future development in the field of large language models.
Unraveling Long Context: Existing Methods, Challenges, and Future Directions
Toronto Machine Learning Series (TMLS) via YouTube
Overview
Syllabus
Unraveling Long Context: Existing Methods, Challenges, and Future Directions
Taught by
Toronto Machine Learning Series (TMLS)
Related Courses
-
Deep Learning for Natural Language Processing
-
Deep Learning for Natural Language Processing
5.0 -
Long-Context Large Language Models - Lecture 15
-
Making LLMs Fully Utilize Context - A Data-Driven Approach
-
How to Code Long-Context LLMs - LongLoRA Implementation with Llama 2 100K
-
LongRoPE and Theta Scaling for Extended Context Length in LLMs - Part 2
