QuickCheck

For more ports, see Wikipedia.

About

QuickCheck is a different kind of unit testing framework. Suppose you want to test a crypto library. For simplicity, assume the encryption algorithm is string reversal. In theory, if you reverse a string, then reverse it again, you will get back the original string. Normally you would hard code a bunch of static assert statements:

asserteq("abc", reverse(reverse("abc")));
asserteq("def", reverse(reverse("def")));
asserteq("123", reverse(reverse("123")));

There's nothing wrong with doing simple assertions, but these only test a handful of cases. Ideally, we want to exhaust every possible input and output and check them for correctness, or at least establish probabilistically that the algorithm is correct. Also, assert statements tend to miss edge cases. What happens if you reverse an empty string?

QuickCheck approaches the problem of program correctness by testing properties. An example of a property would be a function is_even(int) which returns true if an integer is even and false otherwise. QuickCheck is good at throwing a hundred random test cases at a property. If the property fails to hold for some input, QuickCheck halts and prints the offending values.

The syntax for QuickCheck generally looks like:

reversible = function(str) {
   return str == reverse(reverse(str));
}

for_all(reversible, { gen_str })

Output:

*** Failed!
""

The property may fail for "" because the reverse did not properly address the empty string. If we resolve this, the property should hold.

for_all(prop_reversible, { gen_str })

Output:

+++ OK, passed 100 tests.

QuickCheck can test more advanced properties—primitive types are only the beginning.

gen_avl = function() {
   t = gen_tree(gen_node)
   return balance(t);
}

for_all(balanced, gen_tree)

In this way, QuickCheck can test really complicated properties. All you have to do is build a generator. By combining generators, you can build data structures of arbitrary complexity, and have QuickCheck throw a hundred heavy test cases at a property. Properties can be tested over random linked lists, random trees, and random graphs. And many QuickCheck ports are so sophisticated that they can figure out how to generate the data structures themselves, no need to supply manual generator functions. Learn Haskell hint, hint.

For a more in-depth tutorial, read Introduction to QuickCheck.