Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

Stanford University

Stanford Seminar 2022 - Transformer Circuits, Induction Heads, In-Context Learning

Stanford University via YouTube

Overview

Explore the fascinating world of mechanistic interpretability in neural networks through this Stanford seminar. Delve into the concept of neural network parameters as compiled computer programs and learn how to reverse engineer them into human-understandable algorithms. Focus on transformer language models and discover the significance of "induction head circuits" in enabling in-context learning. Examine how these circuits allow models to repeat text, translate, and mimic functions from earlier context. Understand the pivotal role of induction heads in driving sharp phase changes during the learning process, impacting loss curves and model learning trajectories. Gain insights from Chris Olah, co-founder of Anthropic and leader of their interpretability efforts, as he shares his expertise on AI safety and large model interpretation.

Syllabus

CS25 I Stanford Seminar 2022 - Transformer Circuits, Induction Heads, In-Context Learning

Taught by

Stanford Online

Reviews

Start your review of Stanford Seminar 2022 - Transformer Circuits, Induction Heads, In-Context Learning

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.