Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

YouTube

Training Llama 2 in Julia - Scaling Large Language Models

The Julia Programming Language via YouTube

Overview

Discover how to train large language models like Llama(2) using Julia in this JuliaCon 2024 conference talk. Learn about scaling neural network training to multiple GPUs simultaneously using Dagger.jl and Flux.jl. Explore the challenges and solutions in implementing a parallel training pipeline for LLMs, including model and data description, job setup, and performance scaling. Gain insights into fine-tuning pre-trained models with techniques like Low Rank Adaptation (LoRA) for specialized tasks. Understand the components required for efficient large-scale GPU workloads in the Julia ecosystem and the potential applications for various model types beyond LLMs.

Syllabus

Train a Llama(2) in Julia! | Gandhi, P Samaroo | JuliaCon 2024

Taught by

The Julia Programming Language

Reviews

Start your review of Training Llama 2 in Julia - Scaling Large Language Models

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.