Polyglot Data with Python - Introducing Pandas and Apache Arrow

Explore polyglot data handling in Python using Pandas and Apache Arrow in this informative talk from PyCon US. Discover how to overcome challenges in exchanging data between different ecosystems, addressing limitations of Pandas and NumPy outside the Python environment. Learn techniques for efficient data acquisition, manipulation, and exchange without resorting to slow conversion code or unnecessarily large files. Gain insights into working seamlessly in heterogeneous environments, handling data from various sources within Python, and transferring it back to other ecosystems transparently. The presentation covers topics such as CSV scalability issues, data modification, protocols, and the "Pioneer rule," along with practical demonstrations of Pandas Dataframe, Metadata, and File System functionalities.