Standardised Coroutine Concurrency in Python 3

By Robert Smallshire

Coroutines are an old idea undergoing a resurgence in popularity as they facilitate highly concurrent applications without recourse to the complications and intricacies of threads and locks. After many years of third-party support for coroutine-based concurrency in Python, the popular Python language has recently introduced first-class support with many new language features together with a supporting standard library called asyncio.

Coroutines are threads-of-execution implemented via resumable functions which, when combined with a suitable scheduler, can be used to support concurrency in the style of cooperative multitasking which can be simpler to reason about, and easier to debug, than the alternatives.

In the first part of this session we demonstrate the simplicity and essential qualities of coroutine-based concurrency by building a simple concurrency framework from scratch, in Python 3. This efficiently removes any notion that there is any hidden magic in how coroutines work in practice. We then, by a series of simple, mechanical transformations, convert our home-grown example into real Python 3 code using the recently introduced async and await language features, together with the asyncio library. In the final part of the talk we explore some of the abstractions Python 3 builds on top of coroutines such as futures, tasks, transports and asynchronous streams, and demonstrate how to use these in a practical application.

This session can serve as an introduction to coroutine-based concurrency for those new to the idea irrespective of programming language, and window into how asynchronous programming in Python works in theory and in practice. Around 80% of the session is code and live demonstration, with 20% supporting material.