Completed
Chunking: loading Numpy chunks with Zarr
Class Central Classrooms beta
YouTube videos curated by Class Central.
Classroom Contents
Small Big Data - Using NumPy and Pandas When Your Data Doesn't Fit in Memory
Automatically move to the next video in the Classroom when playback concludes
- 1 Small Big Data
- 2 Prelude: the most important question
- 3 TIME FOR A BIG DATA CLUSTER!!!!
- 4 A non-solution: don't use RAM, just disk
- 5 The software solution: use less RAM
- 6 Compression: Numpy dtypes
- 7 Compression: sparse arrays
- 8 Compression: Pandas dtypes When loading data you can specify types
- 9 Chunking: loading Numpy chunks with Zarr
- 10 Chunking: with Pandas
- 11 Indexing: the simplest solution
- 12 Indexing: Pandas without indexing
- 13 Indexing: populate SQLite from Pandas
- 14 Indexing: load from SQLite into DataFrame
- 15 Indexing: SQLite vs. CSV
- 16 Conclusion: what about other libraries?
- 17 Conclusion: don't forget about
