Caching for Jupyter Notebooks

Discover effective caching techniques for Jupyter Notebooks in this 29-minute conference talk from EuroPython 2024. Learn why caching is crucial for speeding up development and improving workflow efficiency. Explore various caching methods, including in-memory, on-disk, database, and object storage options. Understand different cache invalidation strategies and security considerations when dealing with protected data. Gain insights into readily available caching tools from the Python standard library, as well as third-party options like IPython magics and cachetools. Follow along as the speaker demonstrates how to build a custom mini-caching framework tailored to specific use cases, using pandas and Spark as examples. Recognize when caching efforts may become excessive and learn to maintain a balanced approach. This talk is suitable for developers and data scientists of all experience levels looking to optimize their Jupyter Notebook workflows.