Explore Microsoft's groundbreaking 17-billion parameter language model and the innovative ZeRO optimizer in this informative video. Dive into the technical details of how ZeRO enables efficient model and data parallelism without sacrificing training speed. Learn about the Turing-NLG model's state-of-the-art perplexity achievements and the DeepSpeed framework that powers it. Gain insights into the latest advancements in large-scale language model training and optimization techniques that are pushing the boundaries of natural language processing.
Overview
Syllabus
Turing-NLG, DeepSpeed and the ZeRO optimizer
Taught by
Yannic Kilcher
Related Courses
-
Sentiment Analysis with Deep Learning using BERT
-
Ultimate Guide to Scaling ML Models - Megatron-LM - ZeRO - DeepSpeed - Mixed Precision
-
Generative AI with Large Language Models
-
Create a Large Language Model from Scratch with Python – Tutorial
-
Add Natural Language Processing AI power to App by LUIS API
-
Running BLOOM 176B LLM Inference with AWS ML and DeepSpeed
