Dive into a 23-minute video exploring Kolmogorov-Arnold Networks (KAN), a potential alternative to traditional Multi-Layer Perceptrons (MLPs) that power modern AI systems like ChatGPT, LLAMA, and DALLE. Learn about the mathematical foundations of KAN architectures, including B-splines and complex polynomials, through detailed explanations and practical implementations. Explore key concepts like residual activation functions, spline grids, and fine-grained training approaches while understanding the computational complexity and interpretability aspects of KAN. Compare experimental results between KAN and MLP networks, examining their performance in continual learning scenarios and addressing catastrophic forgetting. Gain insights into choosing between KAN and MLP architectures for different applications, supported by demonstrations using a toy problem and comprehensive implementation examples from the official PyKAN repository.
Kolmogorov-Arnold Networks: Understanding KAN Architecture and Comparison with MLPs
Overview
Syllabus
- Intro
- MLPs vs KAN
- simple polynomials as Complex polynomials
- Toy Problem with KAN
- B-Splines
- KAN implementation Residual Activation Function
- Spline Grids and grid extension
- Fine grained training
- computational complexity
- Interpretability
- Experiments and results
- continual learning and catastrophic forgetting
- Choosing between KAN and MLP
Taught by
AI Bites
