Modern Data Science with Vaex - A New Approach to DataFrames and Pipelines

Explore modern data science techniques using Vaex, a powerful DataFrame library, in this 51-minute EuroPython Conference talk. Learn how to efficiently process large datasets on personal computers by leveraging computational graphs, lazy evaluations, memory-mapped storage, and out-of-core algorithms. Discover methods for cleaning, filtering, grouping, and transforming data while visualizing and analyzing correlations. Gain insights into handling datasets with millions or billions of samples without relying on distributed computing. Follow along as the speaker demonstrates practical examples using New York City taxi data, covering topics such as expressions, memory mapping, missing values, filtering, categorizing, group operations, density maps, machine learning, and virtual columns. Understand how Vaex optimizes memory and CPU usage, enabling data scientists to work effectively on laptops or workstations with limited RAM but fast SSD storage.