Explore the evolution and impact of deep learning in speech recognition through this Stanford seminar. Trace the history of artificial intelligence from early milestones like Arthur Samuel's checkers program to modern breakthroughs in deep learning. Examine key concepts including perceptron learning, loss functions, stochastic gradient descent, and multi-layer perceptrons. Delve into the fundamental equation of speech recognition, covering language models and acoustic models. Investigate the resurgence of neural networks in speech recognition, the development of deep belief networks, and their applications in face recognition. Gain insights into real-world implementations of deep learning in speech technology, including Apple's Siri architecture, hands-free voice activation, dictation systems, and voicemail transcription.
Overview
Syllabus
Introduction.
Birth of Artificial Intelligence.
Checkers (Arthur Samuel, 1956).
ELIZA (Weizenbaum 1966).
2001 Space Odyssey (Stanley Kubrick, 1968).
Deep Blue (IBM, 1997).
Deep Learning (Hinton, 2006).
Jeopardy (IBM, 2011).
The imitation game (2014).
Improve on Task T with respect to performance metric P based on experience E.
Perceptron Learning (Rosenblatt, 1957).
A probabilistic framework.
Loss function Loss function between two probability distributions.
Stochastic gradient descent.
N-ary classification.
Multi-layer Perceptron (Werbos, 1974).
Binary Classification Tasks.
Fundamental Equation of Speech Recognition.
Language Model.
Acoustic Model (Hidden Markov Models) HUT.
Neural Networks for Speech Recognition in the 1990s.
Neural Network Winter for Speech Recogntion.
Open Challenge Tasks (DARPA).
Deep Belief Networks = Deep Neural Networks.
Deep Learning for Speech (Deng et al., 2010).
Deep Neural Networks: What was new?.
DNN on Face Images (2012) Deep Belief Net on Face Images.
Deep Learning in Speech Recognition.
Machine Learning across Apple Products.
Siri Architecture.
Hands-Free Siri.
Dictation.
Voicemail transcription.
Taught by
Stanford Online
