How to use criterion with stack

Question

I set up a simple stack project, and a .cabal entry for the benchmark tests:

benchmark leaves-of-a-tree-bench
  type:             exitcode-stdio-1.0
  hs-source-dirs:   src, bench
  main-is:          MainBenchmarkSuite.hs
  build-depends:    base
                  , criterion
                  , random
                  , leaves-of-a-tree
  ghc-options:      -Wall
                    -O2
  default-language: Haskell2010

However after running stack bench I get the following error:

setup-Simple-Cabal-1.22.5.0-ghc-7.10.3: Error: Could not find benchmark program      
".stack-work/dist/x86_64-linux/Cabal-1.22.5.0/build/leaves-of-a-tree-bench/leaves-of-a-tree-bench".
Did you build the package first?

Am I missing something?

EDIT: I uploaded the project to a github repository

Can you upload a full project with this problem somewhere? (Unrelated: by defining `leaves-of-a-tree` in your `build-depends` and adding `src` to `hs-source-dirs` you are depending on your library component twice) — Jan Gerlinger, May 27 '16 at 16:01
I did try `stack build`. The problem is that the benchmark program is not being found. @Jan Thanks for the remark. I also uploaded the relevant files to github. — Damian Nadales, May 28 '16 at 06:37

score 2 · Accepted Answer · answered May 28 '16 at 16:24

There is some weird cabal stuff going on here.

Your LeavesOfATreeBench.hs:

-- |

module LeavesOfATreeBench where

import           Criterion.Main
import           Data.BinTree

mkTree :: [a] -> BinTree a
mkTree [] = Nil
mkTree (x:xs) = Fork x (mkTree lxs) (mkTree rxs)
  where (lxs, rxs) = splitAt ((length xs + 1) `div` 2) xs

main :: IO ()
main = defaultMain [
  bgroup "leaves"
    [ bench "tree 0" $ whnf leaves (mkTree ([0 .. 20] :: [Integer]))
    , bench "tree 1" $ whnf leaves (mkTree ([0 .. 200] :: [Integer]))
    ]
  ]

Now, once I simply remove the line

module LeavesOfATreeBench where

everything works as expected:

Registering leaves-of-a-tree-0.1.0.0...
leaves-of-a-tree-0.1.0.0: benchmarks
Running 1 benchmarks...
Benchmark leaves-of-a-tree-bench: RUNNING...
benchmarking leaves/tree 0
time                 41.81 ns   (41.51 ns .. 42.29 ns)
                     0.999 R²   (0.998 R² .. 0.999 R²)
mean                 42.65 ns   (42.12 ns .. 43.50 ns)
std dev              2.293 ns   (1.526 ns .. 3.690 ns)
variance introduced by outliers: 75% (severely inflated)

benchmarking leaves/tree 1
time                 71.11 ns   (70.41 ns .. 71.84 ns)
                     0.999 R²   (0.999 R² .. 0.999 R²)
mean                 71.30 ns   (70.45 ns .. 72.19 ns)
std dev              2.917 ns   (2.431 ns .. 3.507 ns)
variance introduced by outliers: 62% (severely inflated)

Benchmark leaves-of-a-tree-bench: FINISH
Completed 2 action(s).

As Stack is using the Cabal library for most build stuff, I would guess this is Cabal-related. If it's not considered a bug, at least the documentation should cover this. — Jan Gerlinger, May 30 '16 at 20:13

score 0 · Answer 2 · answered May 28 '16 at 03:18

0

This reminds me of what happens when you don't have a proper Main module. Do you have "module MainBenchmarkSuite" at the top?

I can't find the ghc ticket for this, but for some reason ghc treats this as only a warning. IIRC it's resolved in GHC 8.0

answered May 28 '16 at 03:18

mgsloan

3,245
21
20

I think everything is alright with the module containing the benchmark program. However it might be that I'm missing something. I added a link to the github repository that contains the project. Please note that I did some renaming while trying to look for the solution. – Damian Nadales May 28 '16 at 06:39

How to use criterion with stack

2 Answers2