Explore the development and capabilities of GPT-NeoX-20B, a 20 billion parameter open-source language model, in this insightful interview with EleutherAI co-founder Connor Leahy. Discover the process of training, hardware acquisition, and model performance. Learn about the differences between GPT-Neo, GPT-J, and GPT-NeoX, and gain insights into the challenges of training large language models. Find out how to try the model yourself using GooseAI and hear final thoughts on the project's impact and future potential.
GPT-NeoX-20B - Open-Source Huge Language Model by EleutherAI - Interview With Co-Founder Connor Leahy
Overview
Syllabus
- Intro
- Start of interview
- How did you get all the hardware?
- What's the scale of this model?
- A look into the experimental results
- Why are there GPT-Neo, GPT-J, and GPT-NeoX?
- How difficult is training these big models?
- Try out the model on GooseAI
- Final thoughts
Taught by
Yannic Kilcher
Related Courses
-
Create a Large Language Model from Scratch with Python – Tutorial
-
Open Source LLMOps Solutions
-
How EleutherAI Trains and Releases LLMs - Interview with Stella Biderman
-
CM3- A Causal Masked Multimodal Model of the Internet
-
One Model for All the Tasks - BLIP - Author Interview
-
Author Interview - Memory-Assisted Prompt Editing to Improve GPT-3 After Deployment
