2

Tasty has only the function withResource to manage resources in your tests.

The function takes a resource initialization and cleanup function as arguments:

withResource :: IO a -> (a -> IO ()) -> TestTree -> TestTree

I'm trying to test a Servant application, so I'd like to use Warp.testWithApplication per servant's testing tutorial, but this is a with*-style function (one that calls your action and manages the resource for you), i.e. the type is:

testWithApplication :: IO Application -> (Port -> IO a) -> IO a

am I missing something or is it really hard to glue the two together given that I don't have an initializer/cleanup function but a wrapper with... function?

Note

I know I can just wrap the whole test-suite with testWithApplication (in main) but I'd prefer if the API was started only for the TestTree that requires it.

fghibellini
  • 636
  • 4
  • 15
  • There does seem to be a mismatch. In tasty-based unit testing libraries, assertions are usually just `IO` actions https://hackage.haskell.org/package/tasty-hunit-0.10.0.3/docs/Test-Tasty-HUnit.html#t:Assertion so the simplest solution could perhaps be to invoke `testWithApplication` inside each individual test. – danidiaz Jun 13 '21 at 12:11

1 Answers1

1

I know I can just wrap the whole test-suite with testWithApplication (in main) but I'd prefer if the API was started only for the TestTree that requires it.

Right, so there are two different ways to achieve this.

One way is what the linked tutorial does with Hspec's around: start and stop your app for each individual test.

Tasty doesn't provide a separate utility for this use case because it's fairly trivial to do it yourself: simply call testWithApplication inside each testCase (or you could define an alias for a composition of testCase and testWithApplication to save you some typing). But if you have multiple tests for the same application, you might think it's a bit wasteful to start and stop it for each individual test.

That's where tasty's withResource (or HSpec's aroundAll) comes in. When you use it, the app is started right before the first test that needs it and stopped right after the last such test.

But that relies crucially on the dynamic scope of resources — i.e. we don't know in advance in what order the resources will be allocated or de-allocated. (This may seem surprising if you think of executing the test suite as traversing the test tree and running all tests in order. But the reality is more complicated due to parallel test execution and dependencies between tests, so the actual execution order can be arbitrary.)

And that dynamic scope is incompatible with the "with"-style functions, which normally only create nested, stack-like scopes.

So your options are:

  1. Start an app at the top level and have it running for the whole duration of the test suite (what you describe in your question).
  2. Start and stop the app at the bottom level (i.e. within each testCase), which is what the linked tutorial does with Hspec.
  3. Don't use testWithApplication, but instead write separate functions to start and stop the app. The appeal of testWithApplication compared to separate acquire/release functions is that it handles exceptions for you and ensures the the release action will run, but tasty already takes care of that.
  4. Finally, there's a trick I describe in Decompose ContT that uses threads to achieve dynamic scope with "with"-style functions (which are equivalent to ContT values). This could be the most practical solution if decomposing testWithApplication into separate acquire/release actions is too much of a hassle. HSpec's aroundAll also seems to be using something similar and threads-based.
Roman Cheplyaka
  • 37,738
  • 7
  • 72
  • 121