Explore models of dialogue in this comprehensive lecture from CMU's Neural Nets for NLP 2018 course. Delve into various types of dialogue, generation-based models, and neural approaches for dialogue response generation. Learn about the importance of global coherence in dialogue and solutions like using standard architectures with more context. Examine discourse-level VAE models, diversity-promoting objectives, and methods for evaluating dialogue systems. Discover techniques for infusing personality into dialogue agents and explore retrieval-based chat systems. Gain insights into neural approaches for natural language understanding, slot filling, dialog state tracking, and language generation from dialog states.
Overview
Syllabus
Types of Dialog
Two Paradigms
Generation-based Models (Ritter et al. 2011)
Neural Models for Dialog Response Generation
Dialog More Dependent on Global Coherence
One Solution: Use Standard Architecture w/ More Context
Discourse-level VAE Model (Zhao et al. 2017)
Diversity Promoting Objective for Conversation (Li et al. 2016) • Basic idea we want responses that are likely given the context, unlikely otherwise • Method: subtract weighted unconditioned log probability from conditioned probability (calculated only on first few words)
Using Multiple References with Human Evaluation Scores (Gallay et al. 2015)
Learning to Evaluate • Use context, true response, and actual response to learn a regressor that predicts goodness (Lowe et al. 2017)
Problem 3: Dialog Agents should have Personality
Personality Infused Dialog (Mairesse et al. 2007)
Dialog Response Retrieval
Retrieval-based Chat (Lee et al. 2009)
Neural Response Retrieval (Nio et al. 2014)
Smart Reply for Email Retrieval (Kannan et al. 2016)
NLU (for Slot Filling) w/ Neural Nets (Mesnil et al. 2015)
Dialog State Tracking
Language Generation from Dialog State w/ Neural Nets (Won et al. 2015)
Taught by
Graham Neubig
