Explore advanced search algorithms for natural language processing in this lecture from CMU's Neural Networks for NLP course. Dive into beam search and A*-type search techniques, examining their benefits, drawbacks, and applications in neural parsing and machine translation. Learn about threshold-based pruning, normalization methods, and strategies for dealing with action disparities. Investigate recent research on continuous relaxation of beam search, global neural CCG parsing, and estimating future costs in search algorithms. Gain insights into effective inference methods for generative neural parsing and approaches for improving beam search in training.
Overview
Syllabus
Intro
Potential Problems
Dealing with disparity in actions Effective Inference for Generative Neural Parsing (Mitchell Stern et al., 2017)
Threshold based pruning 'Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation' (Y Wu et al. 2016)
More complicated normalization Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation' (Y Wu et al. 2016)
Beam Search-Benefits and Drawbacks
More beam search in training A Continuous Relaxation of Beam Search for End-to-end Training of Neural Sequence Models (Goyal et al., 2017)
Classical A* parsing al., 2003
Is the heuristic admissible? Global Neural CCG Parsing with Optimality Guarantees (Lee et al. 2016)
Estimating future costs Learning to Decode for Future Success (Li et al., 2017)
Monte-Carlo Tree Search Human-like Natural Language Generation Using
Taught by
Graham Neubig
Related Courses
-
Neural Nets for NLP 2019 - Advanced Search Algorithms
-
Neural Nets for NLP 2017 - Neural Semantic Parsing
-
Neural Nets for NLP 2020: Advanced Search Algorithms
-
Neural Nets for NLP 2021 - Conditioned Generation
-
Neural Nets for NLP 2017 - Parsing With Dynamic Programs
-
Neural Nets for NLP 2018 - Neural Semantic Parsing
