Explore the development and capabilities of GPT-NeoX-20B, a 20 billion parameter open-source language model, in this insightful interview with EleutherAI co-founder Connor Leahy. Discover the process of training, hardware acquisition, and model performance. Learn about the differences between GPT-Neo, GPT-J, and GPT-NeoX, and gain insights into the challenges of training large language models. Find out how to try the model yourself using GooseAI and hear final thoughts on the project's impact and future potential.
GPT-NeoX-20B - Open-Source Huge Language Model by EleutherAI - Interview With Co-Founder Connor Leahy
Overview
Syllabus
- Intro
- Start of interview
- How did you get all the hardware?
- What's the scale of this model?
- A look into the experimental results
- Why are there GPT-Neo, GPT-J, and GPT-NeoX?
- How difficult is training these big models?
- Try out the model on GooseAI
- Final thoughts
Taught by
Yannic Kilcher
Related Courses
-
Create a Large Language Model from Scratch with Python – Tutorial
-
Open Source LLMOps Solutions
-
One Model for All the Tasks - BLIP - Author Interview
-
CM3- A Causal Masked Multimodal Model of the Internet
-
Reproducing GPT-2 (124M) from Scratch - Implementation and Optimization
-
Meta's Llama 3 - The Next Generation of Open-Source Language Models
