Small Big Data - Using NumPy and Pandas When Your Data Doesn't Fit in Memory

Small Big Data - Using NumPy and Pandas When Your Data Doesn't Fit in Memory

PyCon US via YouTube Direct link

Small Big Data

1 of 17

1 of 17

Small Big Data

Class Central Classrooms beta

YouTube videos curated by Class Central.

Classroom Contents

Small Big Data - Using NumPy and Pandas When Your Data Doesn't Fit in Memory

Automatically move to the next video in the Classroom when playback concludes

  1. 1 Small Big Data
  2. 2 Prelude: the most important question
  3. 3 TIME FOR A BIG DATA CLUSTER!!!!
  4. 4 A non-solution: don't use RAM, just disk
  5. 5 The software solution: use less RAM
  6. 6 Compression: Numpy dtypes
  7. 7 Compression: sparse arrays
  8. 8 Compression: Pandas dtypes When loading data you can specify types
  9. 9 Chunking: loading Numpy chunks with Zarr
  10. 10 Chunking: with Pandas
  11. 11 Indexing: the simplest solution
  12. 12 Indexing: Pandas without indexing
  13. 13 Indexing: populate SQLite from Pandas
  14. 14 Indexing: load from SQLite into DataFrame
  15. 15 Indexing: SQLite vs. CSV
  16. 16 Conclusion: what about other libraries?
  17. 17 Conclusion: don't forget about

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.