CMU Multilingual NLP - Dependency Parsing

Two Types of Linguistic Structure
Why Dependencies?
Universal Dependencies Treebank Standard format for parse trees in many languages
Adding Inductive Bias to Neural Models • Bias self attention to follow syntax
Understanding Language Structure Example of extracting morphological agreement rules using dependency relations
Searching over Parsed Corpora Search using 'syntactic regex'
Analysis of Other Linguistic Phenomena • Examining power and agency in film scripts
Arc Standard Shift-Reduce Parsing (Yamada & Matsumoto 2003, Nivre 2003)
Shift Reduce Example
Classification for Shift-reduce
Encoding Stack Configurations w/ RNNS
Transition-based parsing State embeddings
(First Order) Graph-based Dependency Parsing
Graph-based vs. Transition Based
Chu-Liu-Edmonds (Chu and Liu 1965, Edmonds 1967)
Find the Best Incoming
Subtract the Max for Each
Recursively Call Algorithm
Expand Nodes and Delete Edge Deleted from cycle
Sequence Model Feature Extractors (Kipperwasser and Goldberg 2016)
BiAffine Classifier (Dozat and Manning 2017)
Difficulty In Multilingual Dependency Parsing
Example Improvement 1: Order-insensitive Encoders . Standard cross-lingual transfer can fail with large word order differences between source and target Change model structure to be order-insensitive to avoid over-titting to source
Generative Model Fine-tuning • Use generative model that can be trained unsupervised, and fine-tune on the target language
Example Improvement 3: Linguistically Informed Constraints • Add constraints based on a priori knowledge of the language structure
Discussion Question