Explore deep learning fundamentals in this comprehensive tutorial by Carnegie Mellon University's Ruslan Salakhutdinov. Delve into the impact of deep learning, generative models, and image understanding. Learn about feature representations, traditional approaches, and neural networks. Discover the intricacies of feedforward neural networks, activation functions, and stochastic gradient descent. Examine model selection techniques, early stopping, and optimization tricks. Gain insights into unsupervised pre-training, fine-tuning, and dropout. Understand the inspiration from the visual cortex and why training deep networks can be challenging. Master essential concepts like batch normalization and visualization techniques to enhance your deep learning expertise.
Overview
Syllabus
Intro
Mining for Structure
Impact of Deep Learning
Deep Generative Model
Example: Understanding Images
Caption Generation
Talk Roadmap
Learning Feature Representations
Traditional Approaches
Computer Vision Features
Audio Features
Neural Networks Online Course
Feedforward Neural Networks
Artificial Neuron
Activation Function
Single Hidden Layer Neural Net
Multilayer Neural Net
Capacity of Neural Nets
Universal Approximation
Stochastic Gradient Descend
Computational Flow Graph
Model Selection
Early Stopping
Mini-batch, Momentum
Local vs. Distributed Representations
Inspiration from Visual Cortex
Why Training is Hard
Unsupervised Pre-training
Fine-tuning
Dropout at Test Time
Batch Normalization
Optimization Tricks
Visualization
Taught by
Simons Institute
