Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

YouTube

Symbolic Knowledge Distillation- From General Language Models to Commonsense Models Explained

Yannic Kilcher via YouTube

Overview

Explore an innovative approach to creating commonsense knowledge graphs using large language models in this 45-minute video lecture. Learn how to leverage GPT-3's capabilities through clever prompting techniques and targeted critic models to generate high-quality symbolic knowledge. Discover the process of distilling commonsense knowledge from general language models, and understand how this method outperforms traditional human-generated corpora in quantity, quality, and diversity. Gain insights into the Symbolic Knowledge Distillation framework, its application to the ATOMIC dataset, and its potential to revolutionize the training of smaller, specialized commonsense models. Follow along as the lecture covers topics such as generating events and inferences, evaluating datasets, and using critic models to filter data, ultimately leading to key findings that challenge conventional approaches in NLP and commonsense reasoning.

Syllabus

- Intro & Overview
- Sponsor: Weights & Biases
- Commonsense Knowledge Graphs
- ATOMIC dataset
- Generating the corpus from a model
- Prompting GPT-3
- Generating Events
- Generating Inferences
- Evaluating the created dataset
- Introducing the critic
- Using the critic to filter the data
- Training a student on the generated data
- Key Findings
- Comments & Conclusion

Taught by

Yannic Kilcher

Reviews

Start your review of Symbolic Knowledge Distillation- From General Language Models to Commonsense Models Explained

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.