Explore continual learning and catastrophic forgetting in deep neural networks through this 42-minute lecture. Delve into the context, evaluation methods, and algorithms based on regularization, dynamic architectures, and Complementary Learning Systems. Examine data permutation tasks, incremental task learning, multimodal learning, Learning without Forgetting algorithm, Elastic Weight Consolidation, Progressive Neural Networks, and Generative replay. Gain insights from Northeastern University's CS 7150 Deep Learning course, with references to key research papers in the field. Access accompanying lecture notes for a comprehensive understanding of this crucial topic in machine learning.
Overview
Syllabus
Introduction
Context for continual learning
Training on new data
Catastrophic forgetting
Training from scratch
Replaying training data
Evaluating continual learning
Incremental class learning
Multimodal class learning
Strategies for continual learning
Regularization approaches
Learning without forgetting
Regularization
Elastic Weight Consolidation
Bayesian Learning Perspective
Progressive Neural Networks
generative replay
complementary learning systems
Taught by
Paul Hand
Related Courses
-
Introduction to Neural Networks
-
Mastering Neural Networks and Model Regularization
-
Foundations of Deep Learning and Neural Networks
-
Avoiding Catastrophe - Active Dendrites Enable Multi-Task Learning in Dynamic Environments
-
Deep Learning - Crash Course 2023
-
PyTorch for Deep Learning & Machine Learning – Full Course
Related articles
Reviews
5.0 rating, based on 1 Class Central review
-
Great course, which helps us understand how it differ from courses like reinforcement learning, imitation learning, federated learning and more. It is like how we humans accumulate knowledge though time. Hopefully continual learning would push the boundaries of AGI a lot.
