Benchmark Class Reference

#include <evo/benchmark.h>

Detailed Description

Micro benchmarking class.

• Used to benchmark related blocks of code for comparison
• This can benchmark a function or functor (object with operator()() – note that this method must be const)
  • Functors are useful for "parameterizing" the code being benchmarked (via methods/properties)
• Each benchmark runs a given function/functor repeatedly according to a repeat count and measures time elapsed
• Usage:
  • Call scale() first to automatically set a default repeat count (optional)
  • Call run() for each benchmark to run
  • Call report() to show results and clear stored report – the destructor calls this if there's pending report data
    • Report includes a "DiffBest" column showing the difference between AvgCPU of current call and the best (fastest) call in this report – lower value is better
  • Repeat if needed

Shortcut helpers:

• EVO_BENCH_SETUP()
• EVO_BENCH_RUN()
• EVO_BENCH_RUN2()

Example

#include <evo/benchmark.h>
using namespace evo;

// Function to benchmark
inline void bm_function() {
    for (ulong i = 0; i < 100; ++i)
        printf("%s", "");
}

// Functor to benchmark
struct BM_Class {
    void operator()() const {
        for (ulong i = 0; i < 50; ++i)
            printf("%s", "");
    }
};

int main() {
    // Setup and scale using bm_function
    Benchmark bench;
    bench.scale(bm_function);

    // Benchmark bm_function()
    bench.run("bm_function", bm_function);

    // Benchmark instance of BM_Class
    bench.run("BM_Class", BM_Class());

    return 0;
}

Example output:

Name         Time(nsec)  CPU(nsec)  Count    AvgTime(nsec)  AvgCPU(nsec)  DiffBest(nsec)
bm_function  204529700   203125000  1000000  204.5297       203.125       109.375
BM_Class     96745700    93750000   1000000  96.7457        93.75         0

This is the same benchmarking code using the shortcut macros:

int main() {
    // Setup and scale using bm_function
    EVO_BENCH_SETUP(bm_function, 0);

    // Benchmark bm_function()
    EVO_BENCH_RUN(bm_function);

    // Benchmark an instance of BM_Class
    EVO_BENCH_RUN(BM_Class());

    return 0;
}

Parameterized Example

Here's an example that benchmarks parameterized code using a functor:

#include <evo/benchmark.h>
using namespace evo;

struct BM_Class {
    ulong count;

    // Start with default count=50
    BM_Class() : count(50) {
    }

    // Called by EVO_BENCH_RUN2() to finish benchmark name
    void get_name(String& name) const {
        name << '/' << count;
    }

    // Code to benchmark
    void operator()() const {
        for (ulong i = 0; i < count; ++i)
            printf("%s", "");
    }
};

int main() {
    BM_Class bm_obj;
    EVO_BENCH_SETUP(bm_obj, 0);

    // Benchmark with defaults first
    EVO_BENCH_RUN2(bm_obj);

    // Benchmark other parameter variants
    bm_obj.count = 60;
    EVO_BENCH_RUN2(bm_obj);

    return 0;
}

Example output:

Name       Time(nsec)  CPU(nsec)  Count    AvgTime(nsec)  AvgCPU(nsec)  DiffBest(nsec)
bm_obj/50  83509600    78125000   1000000  83.5096        78.125        0
bm_obj/60  112968900   109375000  1000000  112.9689       109.375       31.25

Public Member Functions
	Benchmark (ulongl default_count=0, ulongl default_warmup_count=0)
	Constructor. More...
	~Benchmark ()
	Destructor. More...
Benchmark &	clear ()
	Clear current report. More...
Benchmark &	report (FmtTable::Type type=FmtTable::tTEXT)
	Write benchmark report to stdout. More...
template<class T >
Benchmark &	report_out (T &out, FmtTable::Type type=FmtTable::tTEXT)
	Write benchmark report to output stream or string. More...
template<class T >
Benchmark &	run (const SubString &name, const T &func, ulongl count, ulongl warmup_count=0)
	Run benchmark on given function/functor. More...
template<class T >
Benchmark &	run (const SubString &name, const T &func)
	Run benchmark on given function/functor with default repeat and wramup counts. More...
template<class T >
ulongl	scale (const T &func, uint factor=1)
	Scale the current default repeat count using given function/functor. More...

Constructor & Destructor Documentation

Benchmark()

Benchmark ( ulongl  default_count = 0, ulongl  default_warmup_count = 0  )

inline

Constructor.

Parameters

default_count	Default repeat count to use
default_warmup_count	Default warmup count to use

~Benchmark()

~Benchmark ( )

inline

Destructor.

• This calls report() if it hasn't been called yet

Member Function Documentation

clear()

Benchmark& clear ( )

inline

Clear current report.

report()

Benchmark& report ( FmtTable::Type  type = FmtTable::tTEXT )

inline

Write benchmark report to stdout.

• Call run() first to run each benchmarks, then call this to show the report – repeat if needed
• This clears current benchmark data to setup for another set of benchmarks

Parameters

type	Output format type – see FmtTable::Type

report_out()

Benchmark& report_out ( T &  out, FmtTable::Type  type = FmtTable::tTEXT  )

inline

Write benchmark report to output stream or string.

• Call run() first to run each benchmarks, then call this to show the report – repeat if needed
• This clears current benchmark data to setup for another set of benchmarks

Template Parameters

T	Output stream/string type (inferred by argument)

Parameters

out	Output stream/string to write to
type	Output format type – see FmtTable::Type

Returns

This

run() [1/2]

Benchmark& run ( const SubString &  name, const T &  func, ulongl  count, ulongl  warmup_count = 0  )

inline

Run benchmark on given function/functor.

• This does the benchmark and records the result
• Call report() to print out results

Template Parameters

T	Function/Functor type – inferred by func argument

Parameters

name	Name to use for report
func	Function or functor to benchmark
count	Repeat count to use as the number of times to call func during benchmark
warmup_count	Number of time to call func as a warmup before doing the benchmark

Returns

This

run() [2/2]

Benchmark& run ( const SubString &  name, const T &  func  )

inline

Run benchmark on given function/functor with default repeat and wramup counts.

• This does the benchmark and records the result
• Call report() to print out results

Template Parameters

T	Function/Functor type – inferred by func argument

Parameters

name	Name to use for report
func	Function or functor to benchmark

Returns

This

scale()

ulongl scale ( const T &  func, uint  factor = 1  )

inline

Scale the current default repeat count using given function/functor.

• This figures out a useful repeat count by calling the given function/functor repeatedly until some measurable time has elapsed
• This sets the default repeat count, overriding the constructor argument

Parameters

func	Function or functor to use to scale
factor	Factor to scale (multiply) the decided repeat count

Returns

New repeat count that was set

---

The documentation for this class was generated from the following file:

• evo/benchmark.h