Explore the world of large language models (LLMs) in this insightful interview with Stella Biderman, Executive Director at EleutherAI and Lead Scientist - Mathematician at Booz Allen Hamilton. Delve into EleutherAI's history, the similarities and differences between LLMs, and learn how to choose the right model for specific outcomes. Gain valuable insights on reinforcement learning, pre-training, and fine-tuning techniques for LLMs. Discover the best GPUs for LLM training and EleutherAI's unique approach to developing these models. Examine the importance of interpretability and mechanistic details in LLMs, and understand the significance of memorization in these powerful AI systems.
How EleutherAI Trains and Releases LLMs - Interview with Stella Biderman
Overview
Syllabus
Intro
EleutherAI History & Intro.
LLMs similarities and differences.
Choosing a model for desired outcome
Reinforcement learning: benefits and challenges
Pre-training and fine-tuning LLMs
Best GPUs for LLMs
EleutherAI’s unique LLM-training
Interpretability’s mechanistic details
LLM memorization understanding importance
Taught by
Weights & Biases
