Neural Nets for NLP 2021 - Attention

Learn about attention mechanisms in neural networks for natural language processing in this comprehensive lecture from CMU's Neural Networks for NLP course. Explore the "Attention is All You Need" paper, improvements to attention techniques, specialized attention varieties, and what neural networks actually attend to. Dive into topics like sentence representations, attention score functions, multi-headed attention, training tricks, and applications to various modalities. Gain insights on incorporating Markov properties, coverage, dictionary probabilities, and handling multiple sources in attention-based models.