QuickBencher is an easy to use library for .Net developers who wants to run performance benchmarks from within their code.

Project state: Scope for initial release implemented and working. Real world experience needed to confirm library usefulness and accuracy. Please download the beta and give it a spin!
Update: Version 0.2 have added possibility to use QuickBencher in automated tests, documented here.

Why?

When time comes to squeeze every bit of performance out of your code you will need to do some benchmarking. But it's cumbersome to do this manually, only using tools in the framework like System.Diagnostics.Stopwatch etc. QuickBencher is a tiny library with the goal of making it much easier for you as a developer to get some performance data out of your code.

Key Features

  • Extremely easy to use - just send the code you want to benchmark as an argument to the Benchmark class
  • Measures the following information on any peace of code. The unit of time is seconds.
    • User processor time - the amount of time spent running code inside the application portion of the process
    • System processor time - the amount of time the process has spent running code inside the operating system core
    • Total processor time - the amount of time that the process has spent utilizing the CPU
    • Elapsed real time - measured by a Stopwatch
  • Run several benchmarks sequentially for comparison
  • Can output benchmark results to Console (default), Debug, or Trace (details..) - or you can programmatically check the results in the returned Measurement objects.
  • Possibility to define custom printers for benchmark output (details..)

Example 1: Simple benchmarking

Measure the time it takes to concatenate a string 100,000 times:

Benchmark.Measure(() => 
{
    string s = string.Empty;
    for (int i = 0; i < 100000; i++)
        s += "a";
});
This will output the following data:

3.712824 1.326009 5.038832 ( 4.598000)
Where the four numbers represent user time, system time, total time and real time in seconds.

Example 2: Compare benchmarks for different approaches

Use the Bm() method to see what iterator is the fastest to access all elements in a huge list - this time also with labels:

// Prepare a huge amount of data
var list = new List<string>();
for (int i = 0; i < 10000000; i++)
    list.Add(i.ToString());
string dummy;

// Measure performance of four different iterators..
Benchmark.Bm(x =>
{
    x.Report("For:", () => 
    { 
        for (int i = 0; i < list.Count; i++) dummy = list[i]; 
    });
    x.Report("While:", () => 
    { 
        var enu = list.GetEnumerator(); 
        while(enu.MoveNext()) dummy = enu.Current; 
    });
    x.Report("Foreach:", () => 
    { 
        foreach (var item in list) dummy = item; 
    });
    x.Report("Lambda:", () => 
    { 
        list.ForEach(s => dummy = s ); 
    });
});
The above code will output something like the following:

user system total real
For: 0.202801 0.000000 0.202801 ( 0.175000)
While: 0.109201 0.000000 0.109201 ( 0.131000)
Foreach: 0.093601 0.000000 0.093601 ( 0.122000)
Lambda: 0.046800 0.000000 0.046800 ( 0.079000)


For even more accurate results there is a Bmbm method you should have a look at.

Remarks

  • The times you get when running benchmarks will vary from one execution to the next. It depends on a lot of factors, like the activity of other processes on the machine. You should always run benchmarks multiple times before you can make a decision about what performs best.
  • QuickBencher is inspired by and modelled after the Ruby Benchmark module created by Gotoken. Someone thought: "We should have something like this in C#", and QuickBencher was born. The Benchmark module is therefore the cause of QuickBencher's kind of unusual but also highly efficient interface.

Last edited Jan 27, 2010 at 4:05 PM by tormaroe, version 37