Scaling Machine Learning Workflows to Big Data with Fugue

Explore scaling machine learning workflows to big data using Fugue in this 29-minute conference talk from KubeCon + CloudNativeCon Europe 2022. Learn how to transition from Pandas to distributed computing frameworks like Spark or Dask without reimplementing code. Discover Fugue's open-source abstraction layer that allows data scientists to write framework-agnostic and scale-agnostic code. Follow along as the speakers demonstrate porting native Python code to Spark or Dask with minimal changes, and witness the scaling of data compute from a single machine to a Spark cluster on Kubernetes. Gain insights into lazy evaluation, partitioning, testing, and decoupling logic from execution in big data workflows.