Explore convolutional neural networks for natural language processing in this comprehensive lecture from CMU's Neural Networks for NLP course. Dive into bag of words and n-grams concepts before examining various applications of convolution in language modeling. Learn about context windows, sentence modeling, and structured convolution techniques. Discover the power of stacked and dilated convolutions in processing linguistic data. Investigate convolutional models for analyzing sentence pairs. Gain insights into pooling strategies, non-linear activation functions, and their impact on language processing tasks. Enhance your understanding of advanced CNN architectures tailored for NLP applications through detailed explanations and practical examples.
Overview
Syllabus
Intro
A First Try: Bag of Words (BOW)
Continuous Bag of Words (CBOW) this movie
What do Our Vectors Represent?
Bag of n-grams hate
Why Bag of n-grams?
2-dimensional Convolutional Networks
CNNs for Sentence Modeling
Standard conv2d Function
Padding
Striding
Pooling . Pooling is like convolution, but calculates some reduction function feature-wise • Max pooling: "Did you see this feature anywhere in the range?" (most common) • Average pooling: How prevalent is this feature over the entire range
Stacked Convolution
Dilated Convolution (e.g. Kalchbrenner et al. 2016) . Gradually increase stride every time step (na reduction in length) sentence
Iterated Dilated Convolution (Strubell+2017) . Multiple iterations of the same stack of dilated convolutions
Non-linear Functions
Which Non-linearity Should I Use?
Taught by
Graham Neubig
Related Courses
-
Neural Nets for NLP 2017 - Convolutional Networks for Text
-
Neural Nets for NLP 2020 - Convolutional Neural Networks for Text
-
Convolutional Neural Networks - Full Stack Deep Learning - Spring 2021
-
Neural Nets for NLP - Recurrent Networks for Sentence or Language Modeling
-
Neural Nets for NLP 2017 - A Simple Exercise - Predicting the Next Word in a Sentence
