Andrej Karpathy Courses

Let's Build GPT - From Scratch, in Code, Spelled Out

Comprehensive tutorial on building a GPT model from scratch, covering key concepts like self-attention, multi-headed attention, and transformer architecture. Includes practical coding examples and insights into modern language models.

• YouTube
• 1 hour 56 minutes
• On-Demand
• Free Video

Building Makemore - Building a WaveNet

Dive deep into building a WaveNet-like convolutional neural network, exploring torch.nn, and understanding the typical deep learning development process through hands-on implementation.

• YouTube
• 56 minutes
• On-Demand
• Free Video

Building Makemore - Becoming a Backprop Ninja

Hands-on tutorial on manual backpropagation through a 2-layer MLP, enhancing understanding of gradient flow in neural networks for confident innovation and debugging.

• YouTube
• 1 hour 55 minutes
• On-Demand
• Free Video

Building Makemore - Activations & Gradients, BatchNorm

Explore MLP internals, activations, gradients, and BatchNorm. Learn to diagnose deep networks, understand training challenges, and implement modern techniques for improved performance.

• YouTube
• 1 hour 56 minutes
• On-Demand
• Free Video

Building Makemore - MLP

Implement a multilayer perceptron character-level language model, covering machine learning basics like model training, hyperparameter tuning, evaluation, and addressing under/overfitting.

• YouTube
• 1 hour 16 minutes
• On-Demand
• Free Video

The Spelled-Out Intro to Language Modeling - Building Makemore

Comprehensive tutorial on building a bigram character-level language model, covering tensor operations, neural networks, and PyTorch fundamentals for natural language processing tasks.

• YouTube
• 1 hour 58 minutes
• On-Demand
• Free Video

Intro to Neural Networks and Backpropagation - Building Micrograd

Step-by-step guide to neural networks and backpropagation, covering core concepts, implementation in Python, and practical exercises. Ideal for beginners with basic Python knowledge.

• YouTube
• 2 hours 26 minutes
• On-Demand
• Free Video

Building the GPT Tokenizer - From Strings to Tokens and Back

Comprehensive guide to building a GPT tokenizer from scratch, covering Unicode, Byte Pair Encoding, implementation details, and practical considerations for large language models.

• YouTube
• 2 hours 14 minutes
• On-Demand
• Free Video

Introduction to Large Language Models - Future and Security Challenges

Comprehensive introduction to Large Language Models, covering their functionality, training, future potential, and security challenges. Explores applications, scaling, tool use, and emerging paradigms in AI.

• YouTube
• 1 hour
• On-Demand
• Free Video

Reproducing GPT-2 (124M) from Scratch - Implementation and Optimization

Comprehensive tutorial on reproducing GPT-2 (124M) from scratch, covering implementation, optimization, training setup, and results analysis. Includes practical coding and performance tuning tips.

• YouTube
• 4 hours 1 minute
• On-Demand
• Free Video

Deep Dive into Large Language Models: From ChatGPT to Training and Applications

Dive into a comprehensive exploration of Large Language Models, from fundamental concepts to advanced topics like RLHF, with insights from an industry expert on their development and practical applications.

• YouTube
• 3 hours 31 minutes
• On-Demand
• Free Video