Explore a comprehensive seminar presentation from Harvard CMSA's New Technologies in Mathematics series where University of Montreal researcher Anirudh Goyal delves into the evolution of large language models (LLMs) from basic word prediction to advanced mathematical problem-solving capabilities. Discover how scaling laws govern the emergence of new skills in LLMs as their parameters and training data expand, and learn about the challenges in developing mathematical explanations for this phenomenon. Examine how LLMs develop metacognitive abilities to categorize and label mathematical problems, enabling more effective skill-based prompting approaches. Gain insights into a novel framework that combines human oversight with LLM capabilities to generate challenging mathematical questions, leading to the development of the MATH^2 dataset designed to enhance both artificial and human mathematical reasoning abilities.
From Word Prediction to Complex Skills: Data Flywheels for Mathematical Reasoning
Harvard CMSA via YouTube
Overview
Syllabus
Anirudh Goyal | From Word Prediction to Complex Skills: Data Flywheels for Mathematical Reasoning
Taught by
Harvard CMSA
Related Courses
-
From Word Prediction to Complex Skills: Compositional Thinking and Metacognition in Large Language Models - Keynote
-
GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models
-
Emergence and Reasoning in Large Language Models
-
Inside the Black Box of AI Reasoning - Understanding LLM Knowledge and Complex Problem-Solving
-
LLM Reasoning Limitations: Understanding Linear Order and Logical Hallucinations
-
Peano: Learning Formal Mathematical Reasoning Without Human Data
