The Inadequacy of the Mode in Neural Machine Translation

Explore the limitations of mode-seeking search in neural machine translation (NMT) through this insightful lecture by Wilker Aziz from the University of Amsterdam. Delve into recent findings that challenge the assumption that the most likely sequence is the best translation, and examine various pathologies and biases observed in NMT models. Discover why the empty sequence often emerges as the true most likely output in state-of-the-art NMT systems. Gain a fresh perspective on the probabilistic formulation of NMT and maximum likelihood estimation, focusing on the inadequacy of mode-seeking search methods like beam search. Learn how NMT models distribute probability mass across numerous translations and why the most probable translation is frequently a rare event. Investigate the potential of decision rules that consider the entire probability distribution rather than just its mode. Examine a practical example of such a decision rule and understand why this approach represents a promising direction for future research in the field of neural machine translation.