How to run an individual test with Stack and Haskell Test.Framework?

Question

I'm cloning the following repository and making one change to the stack.yaml by adding at the end:

docker:
    enable: true

To run all the tests for haskoin-core I'm using

stack test haskoin-core:test-haskoin-core

What I want to do is run just one test. If this were HSpec (which it is not) I'd be running something like:

stack test --test-arguments -m "Network.Haskoin.Network.Units"

Now what I could do is modify the file haskcoin-core/test/Main.hs and comment out all the tests I don't want to run. But ya know - there should be a simpler way to run it with just command line parameters. (Mutating the file system goes agains the whole functional grain of Haskell).

I'd also be open to running it with stack ghci somehow.

My question is: How to run an individual test with Stack and Haskell Test.Framework?

Try running `stack test --test-arguments "--help"` to discover what kind of options you can pass to the test suite. — sjakobi, Dec 09 '17 at 18:44

score 5 · Answer 1 · answered Aug 14 '19 at 22:21

5

As of mid 2019, I think stack has changed.

See --help for options:

stack test --test-arguments "--help"
Do a --dry-run to see what tests will run:

stack test --test-arguments "--dry-run"
Select tests with --match. Note, this alleges glob patterns will work but they don't seem to for me:

stack test --test-arguments "--match=foobar"

AFAICT, "foobar" is interpreted like a glob of *foobar*, but I can't be explicit about it.

answered Aug 14 '19 at 22:21

Josh.F

3,666
2
27
37

AFAICT, `"foobar"` is *not* a glob of `*foobar*`. – petre May 22 '21 at 12:57
Note that if the test name contains whitespace, you need to use `-m "//"` for exact match – Abhijit Sarkar Jul 04 '22 at 11:33

score 2 · Accepted Answer · answered Dec 10 '17 at 03:29

Thanks to @sjakobi for this answer.

The process is - list the available test commands:

stack test --test-arguments "--help" haskoin-core:test-haskoin-core

This gives the following result:

haskoin-core-0.4.2: test (suite: test-haskoin-core, args: --help)

Usage: test-haskoin-core [OPTIONS]
                   --help                                       show this help message
  -j NUMBER        --threads=NUMBER                             number of threads to use to run tests
                   --test-seed=NUMBER|random                    default seed for test random number generator
  -a NUMBER        --maximum-generated-tests=NUMBER             how many automated tests something like QuickCheck should try, by default
                   --maximum-unsuitable-generated-tests=NUMBER  how many unsuitable candidate tests something like QuickCheck should endure before giving up, by default
  -s NUMBER        --maximum-test-size=NUMBER                   to what size something like QuickCheck should test the properties, by default
  -d NUMBER        --maximum-test-depth=NUMBER                  to what depth something like SmallCheck should test the properties, by default
  -o NUMBER        --timeout=NUMBER                             how many seconds a test should be run for before giving up, by default
                   --no-timeout                                 specifies that tests should be run without a timeout, by default
  -l               --list-tests                                 list available tests but don't run any; useful to guide subsequent --select-tests
  -t TEST-PATTERN  --select-tests=TEST-PATTERN                  only tests that match at least one glob pattern given by an instance of this argument will be run
                   --jxml=FILE                                  write a JUnit XML summary of the output to FILE
                   --jxml-nested                                use nested testsuites to represent groups in JUnit XML (not standards compliant)
                   --plain                                      do not use any ANSI terminal features to display the test run
                   --color                                      use ANSI terminal features to display the test run
                   --hide-successes                             hide sucessful tests, and only show failures


Test suite failure for package haskoin-core-0.4.2
    test-haskoin-core:  exited with: ExitFailure 1
Logs printed to console

From that we can build a command to list the tests:

stack test --test-arguments "--list-tests" haskoin-core:test-haskoin-core

From there we can use this command to glob for a particular test

stack test --test-arguments=--select-tests=Bloom*Filter haskoin-core:test-haskoin-core

Note the * in place of the space, there seems to be some discussion about how to handle spaces in this scenario.

Now this selects the tests we want to run:

haskoin-core-0.4.2: test (suite: test-haskoin-core, args: --select-tests=Bloom*Filter)

Binary encoding and decoding of bloom types:
  BloomFilter: [OK, passed 100 tests]
Bloom Filters:
  Bloom Filter Vector 1: [OK]
  Bloom Filter Vector 2: [OK]
  Bloom Filter Vector 3: [OK]

         Properties  Test Cases  Total      
 Passed  1           3           4          
 Failed  0           0           0          
 Total   1           3           4

How to run an individual test with Stack and Haskell Test.Framework?

2 Answers2

Linked