Black Friday 2024: 25+ Ways to Save on Online Learning
Faster Inference Using Output Predictions with OpenAI and vLLM

Faster Inference Using Output Predictions with OpenAI and vLLM

Trelis Research via YouTube Direct link

Speculative Decoding with vLLM and Llama 8B

6 of 8
Previous
Next

6 of 8

Speculative Decoding with vLLM and Llama 8B

Class Central Classrooms beta

YouTube videos curated by Class Central.

Classroom Contents

Faster Inference Using Output Predictions with OpenAI and vLLM

Automatically move to the next video in the Classroom when playback concludes
  1. 1 OpenAI output predictions, Cursor fast-apply, vLLM speculative decoding
  2. 2 Cursor Fast Apply - how it works
  3. 3 Video Overview
  4. 4 How does speculative decoding work?
  5. 5 Using OpenAI Output Predictions
  6. 6 Speculative Decoding with vLLM and Llama 8B
  7. 7 Speed-up and Costs of Output Predictions
  8. 8 Resources

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Sign up for free
Someone learning on their laptop while sitting on the floor.