Yannic Kilcher Courses

Symbolic Knowledge Distillation- From General Language Models to Commonsense Models Explained

Explore how to distill commonsense knowledge from large language models to create high-quality knowledge graphs and train smaller, specialized models for improved performance in commonsense reasoning tasks.

• YouTube
• 45 minutes
• On-Demand
• Free Video

Topographic VAEs Learn Equivariant Capsules - Machine Learning Research Paper Explained

Explore Topographic VAEs: a novel approach to deep generative models with organized latent variables, bridging topographic organization and equivariance in neural networks for improved feature learning and transformation handling.

• YouTube
• 32 minutes
• On-Demand
• Free Video

Infinite Memory Transformer - Research Paper Explained

Explores innovative Transformer model with unbounded memory, enabling processing of arbitrarily long sequences. Discusses continuous attention mechanisms, sticky memories, and potential applications in language modeling.

• YouTube
• 37 minutes
• On-Demand
• Free Video

Train Short, Test Long - Attention With Linear Biases Enables Input Length Extrapolation

ALiBi: A novel attention mechanism enabling transformers to process longer sequences than trained on. Uses linear biases instead of position encodings, improving efficiency and extrapolation capabilities.

• YouTube
• 31 minutes
• On-Demand
• Free Video

Underspecification Presents Challenges for Credibility in Modern Machine Learning - Paper Explained

Explores underspecification in ML pipelines, where models with similar training performance behave differently in real-world deployment, leading to instability and poor outcomes across various domains.

• YouTube
• 59 minutes
• On-Demand
• Free Video

Language Models Are Open Knowledge Graphs

Explore how pre-trained language models can be used to construct knowledge graphs without human supervision, uncovering new relations and outperforming existing methods in automated KG construction.

• YouTube
• 52 minutes
• On-Demand
• Free Video

Rethinking Attention with Performers

Explore a novel approach to efficient attention mechanisms in Transformers using random positive orthogonal features, enabling linear-time approximation of attention matrices for scalable deep learning architectures.

• YouTube
• 55 minutes
• On-Demand
• Free Video

LambdaNetworks- Modeling Long-Range Interactions Without Attention

Explore LambdaNetworks, a novel approach to modeling long-range interactions in AI without traditional attention mechanisms, offering improved efficiency and performance in computer vision tasks.

• YouTube
• 1 hour
• On-Demand
• Free Video

Descending Through a Crowded Valley - Benchmarking Deep Learning Optimizers

Comprehensive analysis of popular deep learning optimizers, comparing performance across tasks and providing evidence-based recommendations for algorithm selection and parameter tuning.

• YouTube
• 41 minutes
• On-Demand
• Free Video

An Image is Worth 16x16 Words - Transformers for Image Recognition at Scale

Explore how Transformers outperform CNNs in image recognition, delving into Vision Transformer architecture, experimental results, and implications for AI research.

• YouTube
• 30 minutes
• On-Demand
• Free Video

Training More Effective Learned Optimizers, and Using Them to Train Themselves - Paper Explained

Explores a novel approach to machine learning optimization using neural networks, discussing its architecture, training process, and potential to revolutionize model training through self-improvement and generalization.

• YouTube
• 54 minutes
• On-Demand
• Free Video

The Hardware Lottery - Paper Explained

Explores the "hardware lottery" concept in AI research, discussing how hardware availability influences algorithmic success, with historical examples and future implications for machine learning progress.

• YouTube
• 52 minutes
• On-Demand
• Free Video

Assessing Game Balance with AlphaZero - Exploring Alternative Rule Sets in Chess

Explore how AlphaZero assesses chess variants, analyzing game dynamics and strategic changes resulting from rule modifications. Gain insights into AI's potential for game design and balance evaluation.

• YouTube
• 43 minutes
• On-Demand
• Free Video

Learning to Summarize from Human Feedback

Explore OpenAI's innovative approach to text summarization using human feedback and reinforcement learning, improving summary quality beyond traditional metrics and supervised learning methods.

• YouTube
• 46 minutes
• On-Demand
• Free Video

Self-Classifying MNIST Digits - Paper Explained

Explore how neural cellular automata can achieve global consensus through local communication, applied to digit classification. Discover innovative approaches to modeling biological systems and machine learning tasks.

• YouTube
• 31 minutes
• On-Demand
• Free Video