Learn how to work with dataset builder scripts in Hugging Face Datasets using Python. Explore the download manager, Apache Arrow datatypes, and techniques for creating compressed files. Discover how to generate examples, finish split generators, and add datasets to Hugging Face. Gain insights into using datasets for similarity search, semantic search, vector similarity search, classification, and question-answering tasks. Apply these skills to streamline the training and fine-tuning of models with PyTorch and TensorFlow.
Overview
Syllabus
Intro
Creating Compressed Files
Creating Dataset Build Script
Download Manager
Finishing Split Generator
Generate Examples Method
Add Dataset to Hugging Face
Apache Arrow Features
What's Next?
Taught by
James Briggs