Parallel Computing in Python - Current State and Recent Advances

Explore the current state and recent advances in parallel computing with Python in this EuroPython 2019 conference talk. Gain insights into interfacing Python with parallelism, from leveraging C-extensions to using multiprocessing and multithreading APIs. Learn about high-level parallel processing libraries like concurrent.futures, joblib, and loky, and their applications in various use cases. Discover the latest improvements in the Python standard library, including shared-memory management and serialization enhancements for large Python objects. Understand how these advancements benefit distributed data science frameworks such as dask, ray, and pyspark, and how they address performance bottlenecks in multi-core and multi-machine processing.