Powerful Data Processing Pipelines with C++20 Coroutines
16:00 - 17:30 Friday 8th April 2022 BST
Intermediate
Data processing is at the core of many applications. A useful abstraction for building such systems is the notion of a data pipeline, composed of a number of distinct processing steps. Designing a general solution for building such pipelines turns out to be surprisingly challenging though, in particular if we want to execute the different processing steps concurrently.
We will start by exploring various examples of processing steps and will see how those introduce different requirements that complicate the design of a general solution. We will explore what a universal composable interface for a generic data processing step could look like. We will then motivate how the resulting system can be simplified further by the introduction of coroutines. We will investigate the different ways of how data can be passed between processing steps with C++20 coroutines and arrive at a system that is both easy to use for developers and efficient at runtime.
This talk will be a hands-on exploration with lots of code examples. No prior knowledge of coroutines is required.
Andreas Weis
Andreas has been writing software professionally for more than 10 years, working in areas ranging from real-time graphics to distributed systems to embedded systems. He enjoys hiding complex functionalities behind powerful type-rich interfaces and likes it very much when the compiler finds his bugs for him. Andreas is one of the organizers of the Munich C++ user group, where he gets to discuss these topics with others on a regular basis.
He currently works for Woven Planet, where he works on building modern software for use in safety critical systems.