Benchmarking in C++

By Nikos Athanasiou

A commonplace among talks on benchmarking is the assumption that a stopwatch - a time measuring mechanism already exists. While this may seem a mundane endeavour it is of great importance to have a reliable, cross platform, easy to use framework for performing "benchmarking experiments".

In this talk we’ll walk through the proper use of <chrono> to create such a framework, having the following features:

  • Parametrizable time and clock type per benchmark.

  • Multiple experiments can run on the same benchmark (hence we can group and compare results as we wish).

  • User adjustable sampling size (number of times each experiment will run).

  • Auto detectable sampling (number of times the experiment has to run until we "trust" its results)

  • Benchmarks printable and serializable in a “Python friendly” form.

  • A Python script to read the output of a serialization and make graphs using matplotlib.