Powerful Data Processing Pipelines with C++20 Coroutines

By Andreas Weis

Data processing is at the core of many applications. A useful abstraction for building such systems is the notion of a data pipeline, composed of a number of distinct processing steps. Designing a general solution for building such pipelines turns out to be surprisingly challenging though, in particular if we want to execute the different processing steps concurrently.

We will start by exploring various examples of processing steps and will see how those introduce different requirements that complicate the design of a general solution. We will explore what a universal composable interface for a generic data processing step could look like. We will then motivate how the resulting system can be simplified further by the introduction of coroutines. We will investigate the different ways of how data can be passed between processing steps with C++20 coroutines and arrive at a system that is both easy to use for developers and efficient at runtime.

This talk will be a hands-on exploration with lots of code examples. No prior knowledge of coroutines is required.





Your Privacy

By clicking "Accept Non-Essential Cookies" you agree ACCU can store non-essential cookies on your device and disclose information in accordance with our Privacy Policy and Cookie Policy.

Current Setting: Non-Essential Cookies REJECTED


By clicking "Include Third Party Content" you agree ACCU can forward your IP address to third-party sites (such as YouTube) to enhance the information presented on this site, and that third-party sites may store cookies on your device.

Current Setting: Third Party Content EXCLUDED



Settings can be changed at any time from the Cookie Policy page.