{"id":101,"date":"2011-09-28T23:46:37","date_gmt":"2011-09-28T21:46:37","guid":{"rendered":"http:\/\/ken.friislarsen.net\/blog\/?p=101"},"modified":"2011-09-29T14:16:51","modified_gmt":"2011-09-29T12:16:51","slug":"quality-assessment-haskell-program","status":"publish","type":"post","link":"http:\/\/ken.friislarsen.net\/blog\/2011\/09\/28\/quality-assessment-haskell-program\/","title":{"rendered":"Quality assessment of Haskell programs"},"content":{"rendered":"<p>One of the greatest things about writing code in Haskell is the wonderful libraries (incidently, one of the worst things about writing code in Haskell is the libraries). In particular the libraries for assessing the quality of your own code. I\u2019m especially found of:<\/p>\n<ul>\n<li>\n<p><a href=\"http:\/\/hackage.haskell.org\/package\/QuickCheck\" title=\"QuickCheck\">QuickCheck<\/a> for a quick testing of the correctness of my code, by letting me specify the <em>properties<\/em> I want to check, and taking care of generating random test-cases.<\/p>\n<\/li>\n<li>\n<p><a href=\"http:\/\/hackage.haskell.org\/package\/criterion\" title=\"Criterion\">Criterion<\/a> for checking the time performance of my code in a statistical sound manner. Taking care of dealing with the garbage collector and such like.<\/p>\n<\/li>\n<\/ul>\n<p>In this post I give a quick tour of how to use these libraries.<\/p>\n<h3 id=\"getting-the-libraries\">Getting the libraries<\/h3>\n<p>Our first step is to install QuickCheck and Criterion:<\/p>\n<pre><code>$ cabal install criterion quickcheck\r\n<\/code><\/pre>\n<p>Now we are ready to go.<\/p>\n<h3 id=\"backstory\">Backstory<\/h3>\n<p>The standard list-based implementation of quicksort that you so often see, seems ripe for some optimisations:<\/p>\n<pre><code>quicksort :: Ord a =&gt; [a] -&gt; [a]\r\nquicksort []     = []\r\nquicksort (p:xs) = quicksort lesser ++ [p] ++ quicksort greater\r\n    where\r\n        lesser  = [ x | x &lt;- xs, x &lt; p]\r\n        greater = [ x | x &lt;- xs, x &gt;= p]\r\n<\/code><\/pre>\n<p>One obvious optimisation could be to only traverse <code>xs<\/code> once, when partitioning the elements into lesser and greater (and equal) than <code>p<\/code>. Here is one implementation that uses <code>foldl'<\/code> for maximum performance (we hope):<\/p>\n<pre><code>kquicksort :: Ord a =&gt; [a] -&gt; [a]\r\nkquicksort []     = []\r\nkquicksort (p:xs) = kquicksort lesser ++ equal ++ kquicksort greater\r\n    where\r\n        (lesser,equal,greater) = foldl' part ([],[p],[]) xs\r\n        part (l,e,g) x =\r\n          case compare x p of\r\n            LT -&gt; (x : l, e, g)\r\n            GT -&gt; (l, e, x : g)\r\n            EQ -&gt; (l, x : e, g)\r\n<\/code><\/pre>\n<h3 id=\"checking-some-properties\">Checking some properties<\/h3>\n<p>The basic usage of QuickCheck is to specify the properties you care about as ordinary Haskell functions, and then you use the <code>quickCheck<\/code> function to check the properties. In the following I use <code>QC<\/code> as the qualified name for values from the QuickCheck library.<\/p>\n<p>The first property we want to check is that <code>kquicksort<\/code> returns a sorted result. To check this property we also define a helper predicate <code>sorted<\/code> that checks that a list is sorted:<\/p>\n<pre><code>sorted :: Ord a =&gt; [a] -&gt; Bool\r\nsorted (x1:x2:xs) = x1 &lt;= x2 &amp;&amp; (sorted $ x2:xs) \r\nsorted _          = True\r\n\r\nprop_sorted :: Ord a =&gt; [a] -&gt; Bool\r\nprop_sorted xs = sorted $ kquicksort xs\r\n<\/code><\/pre>\n<p>The next properties that we want to check could be that <code>kquicksort<\/code> is idempotent and if it is given an ordered argument it doesn\u2019t mess it up:<\/p>\n<pre><code>prop_idempotent :: Ord a =&gt; [a] -&gt; Bool\r\nprop_idempotent xs = kquicksort xs == (kquicksort $ kquicksort xs)\r\n\r\nprop_ordered :: Ord a =&gt; QC.OrderedList a -&gt; Bool\r\nprop_ordered (QC.Ordered xs) = xs == kquicksort xs\r\n<\/code><\/pre>\n<p>For <code>prop_ordered<\/code> we use the type class <code>OrderedList<\/code> to specify that this property should only hold for arguments that are sorted.<\/p>\n<p>Now we can use the <code>quickCheck<\/code> function to check all our properties:<\/p>\n<pre><code>checkAll = do\r\n  QC.quickCheck (prop_sorted :: [Int] -&gt; Bool)\r\n  QC.quickCheck (prop_idempotent :: [Int] -&gt; Bool)\r\n  QC.quickCheck (prop_ordered :: QC.OrderedList Int -&gt; Bool)\r\n<\/code><\/pre>\n<p>The type constraints on all the properties are necessary, because that is what makes overloading works, so that <code>quickCheck<\/code> can generate random test-cases. Alternatively we can give our properties a less general non-polymorphic type. However, by keeping the more general type for our properties we can check them for other types as well. For instance, we might want to check it for a custom type, like the following type <code>Colour<\/code>:<\/p>\n<pre><code>data Colour = Red | Green | Blue\r\n            deriving (Eq, Show, Ord)\r\n<\/code><\/pre>\n<p>Before we can use QuickCheck with the <code>Colour<\/code> type, we need to specify how to generate random values of this type. That is, make it an instance of the <code>Arbitrary<\/code> type class:<\/p>\n<pre><code>instance QC.Arbitrary Colour where\r\n  arbitrary = QC.elements [Red, Green, Blue]\r\n<\/code><\/pre>\n<p>Now, we can check the <code>prop_sorted<\/code> property by adding a simple type constraint:<\/p>\n<pre><code>QC.quickCheck (prop_sorted :: [Colour] -&gt; Bool)\r\n<\/code><\/pre>\n<h3 id=\"checking-the-performance\">Checking the performance<\/h3>\n<p>Once we have convinced our-self the code could be correct, we can start worrying about performance. Thus, we turn to Criterion. When doing performance testing there are all sorts of gnarly details that can go wrong and ruin our experiments: we forget to force the lazy evaluation and thus measure the wrong thing, the running time of function we try to measure is to short to get a meaningful reading of with clock resolution on the computer we use, the garbage collector might introduce noise from one sampling into another, we might be using the computer for something while we run the experiments which could introduce noise, and so on.<\/p>\n<p>In the following I use <code>C<\/code> as the qualified name for values from the Criterion library.<\/p>\n<p>Criterion takes care of all these concerns, and just present us with a simple interface where the only thing we need to specify is which functions we want to run, on what data, and how much we want the result to be evaluated. Thus, to benchmark <code>kquicksort<\/code> on the list <code>[20, 10, 30]<\/code> and get a fully evaluated result, we first use the <code>nf<\/code> (for normal form) function from Criterion to get something <code>Benchmarkable<\/code>:<\/p>\n<pre><code>C.nf kquicksort [20, 10, 30]\r\n<\/code><\/pre>\n<p>When we have something benchmarkable, we use the <code>bench<\/code> function to label it and turn it into a <code>Benchmark<\/code><\/p>\n<pre><code>C.bench &quot;kquickcheck on short list&quot; $ C.nf kquicksort [20, 10, 30]\r\n<\/code><\/pre>\n<p>When we have a list of benchmark we can hand it over to <code>defaultMain<\/code> which makes an excellent <code>main<\/code> body for us that will allow us to configure our benchmark from the command-line without recompiling the program. Without further ado:<\/p>\n<pre><code>makeList :: Int -&gt; [Int]\r\nmakeList n = QC.unGen (QC.vector n) (R.mkStdGen 42) 25\r\n\r\nmain :: IO()\r\nmain = do\r\n  checkAll\r\n  let sizes = [ 10000, 20000, 50000, 75000\r\n              , 100000, 250000, 500000, 1000000]\r\n  let inputs = [(n, makeList n) | n &lt;- sizes]\r\n  let benchmarks = [ C.bench (name ++ show n) $ C.nf sort ns \r\n                   | (n, ns)      &lt;- inputs,\r\n                     (name, sort) &lt;- [(&quot;quicksort &quot;,  quicksort),\r\n                                      (&quot;kquicksort &quot;, kquicksort)]]\r\n  C.defaultMain benchmarks\r\n<\/code><\/pre>\n<p>In the helper function <code>makeList<\/code> I have reused the generator framework from QuickCheck for generating some random test data for my benchmark. For simple integer lists as we have here it is a bit overkill, but for more complex input it can be nice.<\/p>\n<h3 id=\"the-complete-program\">The complete program<\/h3>\n<p>If you want to find out if <code>kquicksort<\/code> really is faster than <code>quicksort<\/code> is here the complete program.<\/p>\n<pre><code>import qualified Criterion.Main as C\r\nimport qualified Test.QuickCheck as QC\r\nimport qualified Test.QuickCheck.Gen as QC\r\n\r\nimport Data.List(foldl')\r\nimport qualified System.Random as R\r\n\r\nquicksort :: Ord a =&gt; [a] -&gt; [a]\r\nquicksort []     = []\r\nquicksort (p:xs) = quicksort lesser ++ [p] ++ quicksort greater\r\n    where\r\n        lesser  = [ x | x &lt;- xs, x &lt; p]\r\n        greater = [ x | x &lt;- xs, x &gt;= p]\r\n\r\nkquicksort :: Ord a =&gt; [a] -&gt; [a]\r\nkquicksort []     = []\r\nkquicksort (p:xs) = kquicksort lesser ++ equal ++ kquicksort greater\r\n    where\r\n        (lesser,equal,greater) = foldl' part ([],[p],[]) xs\r\n        part (l,e,g) x =\r\n          case compare x p of\r\n            LT -&gt; (x : l, e, g)\r\n            GT -&gt; (l, e, x : g)\r\n            EQ -&gt; (l, x : e, g)\r\n\r\nsorted :: Ord a =&gt; [a] -&gt; Bool\r\nsorted (x1:x2:xs) = x1 &lt;= x2 &amp;&amp; (sorted $ x2:xs) \r\nsorted _          = True\r\n\r\nprop_sorted :: Ord a =&gt; [a] -&gt; Bool\r\nprop_sorted xs = sorted $ kquicksort xs\r\n\r\nprop_idempotent :: Ord a =&gt; [a] -&gt; Bool\r\nprop_idempotent xs = kquicksort xs == (kquicksort $ kquicksort xs)\r\n\r\nprop_ordered :: Ord a =&gt; QC.OrderedList a -&gt; Bool\r\nprop_ordered (QC.Ordered xs) = xs == kquicksort xs\r\n\r\ndata Colour = Red | Green | Blue\r\n            deriving (Eq, Show, Ord)\r\n\r\ninstance QC.Arbitrary Colour where\r\n  arbitrary = QC.elements [Red, Green, Blue]\r\n\r\ncheckAll = do\r\n  QC.quickCheck (prop_sorted :: [Int] -&gt; Bool)\r\n  QC.quickCheck (prop_sorted :: [Colour] -&gt; Bool)\r\n  QC.quickCheck (prop_idempotent :: [Int] -&gt; Bool)\r\n  QC.quickCheck (prop_ordered :: QC.OrderedList Int -&gt; Bool)\r\n\r\nmakeList :: Int -&gt; [Int]\r\nmakeList n = QC.unGen (QC.vector n) (R.mkStdGen 42) 25\r\n\r\nmain :: IO()\r\nmain = do\r\n  checkAll\r\n  let sizes = [ 10000, 20000, 50000, 75000\r\n              , 100000, 250000, 500000, 1000000]\r\n  let inputs = [(n, makeList n) | n &lt;- sizes]\r\n  let benchmarks = [ C.bench (name ++ show n) $ C.nf sort ns \r\n                   | (n, ns)      &lt;- inputs,\r\n                     (name, sort) &lt;- [(&quot;quicksort &quot;,  quicksort),\r\n                                      (&quot;kquicksort &quot;, kquicksort)]]\r\n  C.defaultMain benchmarks\r\n<\/code><\/pre>\n<p>Compile it with the command-line<\/p>\n<pre><code>$ ghc -O3 -W --make Quicksort.hs -o Quicksort\r\n<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>One of the greatest things about writing code in Haskell is the wonderful libraries (incidently, one of the worst things about writing code in Haskell is the libraries). In particular the libraries for assessing the quality of your own code. I\u2019m especially found of: QuickCheck for a quick testing of the correctness of my code, <a class=\"read-more\" href=\"http:\/\/ken.friislarsen.net\/blog\/2011\/09\/28\/quality-assessment-haskell-program\/\">[&hellip;]<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4,17],"tags":[],"class_list":["post-101","post","type-post","status-publish","format-standard","hentry","category-coding","category-haskell"],"_links":{"self":[{"href":"http:\/\/ken.friislarsen.net\/blog\/wp-json\/wp\/v2\/posts\/101","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/ken.friislarsen.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/ken.friislarsen.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/ken.friislarsen.net\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/ken.friislarsen.net\/blog\/wp-json\/wp\/v2\/comments?post=101"}],"version-history":[{"count":10,"href":"http:\/\/ken.friislarsen.net\/blog\/wp-json\/wp\/v2\/posts\/101\/revisions"}],"predecessor-version":[{"id":110,"href":"http:\/\/ken.friislarsen.net\/blog\/wp-json\/wp\/v2\/posts\/101\/revisions\/110"}],"wp:attachment":[{"href":"http:\/\/ken.friislarsen.net\/blog\/wp-json\/wp\/v2\/media?parent=101"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/ken.friislarsen.net\/blog\/wp-json\/wp\/v2\/categories?post=101"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/ken.friislarsen.net\/blog\/wp-json\/wp\/v2\/tags?post=101"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}