MIT 6.S191 - Deep Learning Limitations and New Frontiers

Overview

Explore deep learning limitations and emerging frontiers in this lecture from MIT's Introduction to Deep Learning course. Delve into the rise of deep learning, the power of neural networks, and historical perspectives on AI hype. Examine neural networks as function approximators and their capacity for generalization. Investigate adversarial attacks on neural networks and the synthesis of robust adversarial examples. Understand the importance of uncertainty in deep learning and explore Bayesian approaches for quantifying model uncertainty. Learn about multi-task learning using uncertainty and the concept of learning to learn through AutoML. Discover how AutoML works on the cloud and spawns powerful ideas for the future of deep learning.

Syllabus

Intro
T-shirts! Today!
Course Schedule
Final Class Project
The Rise of Deep Learning
Power of Neural Nets
Artificial Intelligence "Hype": Historical Perspective
Rethinking Generalization
Capacity of Deep Neural Networks
Neural Networks as Function Approximators Neural networks are excellent function approximators
Adversarial Attacks on Neural Networks
Synthesizing Robust Adversarial Examples
Neural Network Limitations...
Why Care About Uncertainty?
Bayesian Deep Learning for Uncertainty
Elementwise Dropout for Uncertainty
Model Uncertainty Application
Multi-Task Learning Using Uncertainty
Motivation: Learning to Learn
AutoML: Learning to Learn
AutoML: Model Controller At each stes, the model samples a brand new network
AutoML:The Child Network
AutoML on the Cloud
AutoML Spawns a Powerful Idea