Knowledge Is Embedded in Language Neural Networks but Can They Reason?

Explore the capabilities and limitations of language neural networks in reasoning during this 50-minute lecture by Chris Manning from Stanford University. Delve into the evolution of language models, from n-gram models to modern neural language models like GPT-2. Examine the structure and functionality of transformer models and self-attention mechanisms in sequence modeling. Analyze the strengths of current systems while acknowledging their basic natural language understanding errors. Investigate the concept of reasoning in AI, discussing appropriate structural priors and compositional reasoning trees. Discover emerging research directions, including Neural State Machines, and their potential to enhance AI reasoning abilities. Gain insights into the accuracy of Neural State Machines on visual question-answering tasks like GQA.