Explore a comprehensive conference talk that delves into the world of asynchronous I/O in Python, focusing on the role of asyncio and its relationship with other frameworks like Twisted and Tornado. Learn about the historical context of asynchronous programming in Python, the introduction of PEP 3156, and the current state of asyncio integration. Gain insights into the technical details of selector loops, coroutines, and the benefits of asynchronous programming. Discover the future direction of Twisted, the challenges faced by asyncio as a standard API, and potential solutions. Understand the differences between various frameworks, their interoperability, and the continued relevance of Twisted in the Python ecosystem.
Overview
Syllabus
Intro
Release Manager
Deployed with runners that run many copies using threads or processes
use safely (without race conditions)
Thread memory overhead: 32kB to 8MB per thread
The approach of Twisted, Tornado, asyncio, curio
Identical system call at their core: Selector functions
Selector functions take a list of file descriptors (e.g. sockets, open files) and tell you what is ready for reading or writing
Selector loops can handle thousands of open sockets and events
Data is channeled through a transport to a protocol implementation (e.g. HTTP)
Sending data is queued until the network is ready
Higher density per core No threads required! Concurrency, not parallelism
You're probably waiting on the client or the database
coroutines are a special generator
Repairing library API fragmentation
It should be easy for (Python 3.3 ports of) frameworks like Twisted, Tornado, or even gevent to either adapt the default event loop implementation to their needs using a lightweight adapter or proxy, or to replace the default event loop implementation with an adaptation of their own event loop implementation.
Reducing duplication
For this interoperability to be effective, the preferred direction of adaptation in third party frameworks is to keep the default event loop and adapt it to the framework's API. Ideally all third party frameworks would give up their own event loop implementation in favor of the standard implementation.
asyncio is an apple Twisted is a fruit salad
Tornado is a great example of interoperation
Asynchronous code executed in a synchronous style
Why Twisted is still worth using
Time based releases, taken off our trunk branch
Super easy to make your own protocols!
Established library support
Code review Automated testing Thousands of tests
Python 3.4/3.5 is coming to Windows soon!
Taught by
EuroPython Conference
