Learn about the groundbreaking Orca language model in a 12-minute video that explores how this 13-billion parameter model achieves ChatGPT-level reasoning capabilities. Dive into the technical aspects of instruction tuning, explanation tuning, and system instructions that enable Orca to bridge the gap between open-source models and proprietary AI systems. Explore the innovative dataset collection methods using FLAN-v2, understand the sampling algorithm, and examine the training strategies that make Orca stand out from other open-source models like Vicuna and Alpaca. Discover the evaluation results and understand the current limitations of this promising advancement in AI technology, presented by an experienced Machine Learning Researcher with 15 years of industry experience.
Orca LLM: Bridging the Gap Between ChatGPT and Open Source Models
Overview
Syllabus
- Intro
- Instruction Tuning
- Problems with Instruction Tuning
- Explanation Tuning
- System Instructions
- Dataset Collection and FLAN-v2
- Sampling Algorithm
- Training Strategy
- Evaluation
- Limitations and Conclusion
Taught by
AI Bites
Related Courses
-
Improving Accuracy of LLM Applications
-
Building Multimodal Search and RAG
-
Self-Instruct Fine-Tuning of Large Language Models - Introduction to Alpaca
-
ChatGPT vs Flan-T5: Comparing Proprietary and Free LLMs with Performance Tuning
-
Creating Self-Instruct Data Sets for LLM Fine-Tuning with ChatGPT
-
Custom RAG Pipelines and LLM Fine-Tuning - A Gradient Tutorial
