108

I just got started with F#, which is my first functional language. I have been working quasi-exclusively with C#, and enjoy a lot how F# leads me to re-think how I write code. One aspect I find a bit disorienting is the change in the process of writing code. I have been using TDD for years in C# now, and really appreciate to have unit tests to know where I am at.

So far, my process with F# has been to write some functions, play with them with the interactive console until I am "reasonably" sure they work, and tweak & combine. This works well on small-scale problems like the Euler Project, but I can't imagine building something large that way.

How do people approach unit testing and building a test suite for a F# program? Is there an equivalent to TDD? Any pointers or thoughts are appreciated.

knocte
  • 16,941
  • 11
  • 79
  • 125
Mathias
  • 15,191
  • 9
  • 60
  • 92
  • 1
    http://www.expert-fsharp.com/CodeSamples/Forms/AllItems.aspx?RootFolder=%2fCodeSamples%2fChapter18%2fExample06 shows a simple example of using NUnit with F#. – itowlson Jan 01 '10 at 20:32
  • see http://stackoverflow.com/questions/1468772/how-to-handle-unit-tests-in-f – Mauricio Scheffer Jan 02 '10 at 15:34
  • related: http://stackoverflow.com/questions/5667372/what-unit-testing-frameworks-are-available-for-f/5669263#5669263 (Unquote is far more than a footnote/comment as it is within the answerset on this page) – Ruben Bartelink Oct 07 '13 at 12:52
  • One thing missing in these answers is a proper example of Foq, AutoFixture.AutoFoq and AutoFixture.xUnit allied to F#'s type inference. See http://trelford.com/blog/post/test5.aspx and http://trelford.com/blog/post/fstestlang.aspx for a taster and some day I'll write a proper answer here – Ruben Bartelink Nov 20 '13 at 22:45

7 Answers7

77

Test-driven developers should feel right at home in functional languages like F#: small functions that give deterministically repeatable results lend themselves perfectly to unit tests. There are also capabilities in the F# language that facilitate writing tests. Take, for example, Object Expressions. You can very easily write fakes for functions that take as their input an interface type.

If anything, F# is a first-class object-oriented language and you can use the same tools and tricks that you use when doing TDD in C#. There are also some testing tools written in or specifically for F#:

Matthew Podwysocki wrote a great series on unit testing in functional languages. Uncle Bob also wrote a thought provoking article here.

John Oxley
  • 14,698
  • 18
  • 53
  • 78
Ray Vernagus
  • 6,130
  • 1
  • 24
  • 19
  • 9
    I have also developed (and am actively developing) an F# specific unit testing library called Unquote: http://code.google.com/p/unquote/. It allows you to write test assertions as plain, statically checked F# boolean expressions using F# Quotations and automatically produces nice test failure messages. It works configuration-free with special support for both xUnit.net and NUnit and generally supports any exception based unit testing framework. It even works within FSI sessions allowing seamless migration from interactive testing to formal test suites. – Stephen Swensen Apr 03 '11 at 02:23
  • There's [Pex](http://www.pexforfun.com/) too, though that's a bit more difficult to grok. – Benjol Oct 08 '13 at 04:54
  • 1
    Uncle bob link seems dead – Aage Jul 01 '17 at 05:43
  • 2020 update: To be exhaustive, Expecto https://github.com/haf/expecto/ should be added to the list (see post below by the author). IMO it's the most FP-ish testing framework for F# – Digital Stoic Aug 22 '20 at 16:55
22

I use NUnit, and it doesn't strike me as hard to read or onerous to write:

open NUnit.Framework

[<TestFixture>]
type myFixture() = class

    [<Test>]
    member self.myTest() =
       //test code

end

Since my code is a mix of F# and other .Net languages, I like the fact that I write the unit tests in basically the same fashion and with similar syntax in both F# and C#.

David Glaubman
  • 436
  • 2
  • 5
  • 4
    After reading other responses here,I gave FSUnit a try, and I think it's great. It works well with TestDriven.Net (as does NUnit), encourages a fluid style of writing self-documenting tests and, as Ray states, is "more at home in the F# languages". Not bad for 21 lines of code!(and a couple of layout/naming recommendations). Two quick notes: 1. The precompiled FSUnit DLL did not work for me. Building from source (FsUnit.NUnit-0.9.0.fs) fixed the problem. 2. TestDriven.Net doesn't recognize TextFixture names that look `\`like this\``. Test names using double-tick form are recognized. – David Glaubman Jan 11 '10 at 00:18
14

Have a look at FsCheck, an automatic testing tool for F#, is basically a port of Haskell's QuickCheck. It allows you to provide a specification of the program, in the form of properties that the functions or methods should satisfy, and FsCheck tests that the properties hold in a large number of randomly generated cases.

FsCheck CodePlex Page

FsCheck Author Page

flq
  • 22,247
  • 8
  • 55
  • 77
primodemus
  • 516
  • 2
  • 8
  • Yes, I think FsCheck offers a lot more than traditional unit testing frameworks such as NUnit etc. – Robert Jan 05 '10 at 13:37
11

I think this is a very interesting question that I have wondered about a lot myself. My thoughts so far are only thoughts, so take them for what they are.

I think that the safety net of an automated test suite is too valuable an asset to let go, however alluring that interactive console may be, so I plan to continue writing unit tests as I've always done.

One of the main strengths of .NET is the cross-language capabilities. I know I'm going to be writing F# production code soon, but my plan is to write unit tests in C# to ease my way into what is for me a new language. In this way, I also get to test that what I write in F# will be compatible with C# (and other .NET languages).

With this approach, I understand that there are certain features of F# that I can only use internally in my F# code, but not expose as part of my public API, but I will accept that, just as I accept today that there are certain things C# allows me to express (like uint) that aren't CLS compliant, and so I refrain from using them.

Ruben Bartelink
  • 59,778
  • 26
  • 187
  • 249
Mark Seemann
  • 225,310
  • 48
  • 427
  • 736
  • 2
    How your plan was going? It was easy to test f# code with c# code? I started learning f# and my plan is to write some part of my project in f# and I have the same idea: to write unit tests in c# for f# too. – Peter Porfy Sep 20 '12 at 14:56
  • @Mark any updates? I also am having a hard time getting into flow using TDD with F#. – Scott Nimrod Dec 26 '15 at 20:07
  • 1
    @ScottNimrod Quite a few updates: four of [my Pluralsight courses](http://bit.ly/ploehralsight) are about testing or TDD with F#. You'll also find lots of free recordings of conference talks on [my Lanyrd profile](http://lanyrd.com/profile/ploeh). Finally, there's [my blog](http://blog.ploeh.dk). – Mark Seemann Dec 26 '15 at 21:40
  • 3
    @ScottNimrod I cannot recommend taking the time and/or paying the money to view the complete set of Mark's PS courses highly enough - it'll snap it all together in your head with maximum efficiency. While it may or may not apply to your specific needs, the "A Functional Architecture in F#" also connects up a lot of dots and should also be strongly considered. – Ruben Bartelink Dec 29 '15 at 13:09
11

As dglaubman suggests you can use NUnit. xUnit.net also provides support for this and works well with TestDriven.net. The code looks similar to NUnit tests but without the requirement to wrap the test in a containing type.

#light

// Supply a module name here not a combination of module and namespace, otherwise
// F# cannot resolve individual tests nfrom the UI.
module NBody.DomainModel.FSharp.Tests

open System
open Xunit

open Internal

[<Fact>]
let CreateOctantBoundaryReordersMinMax() =
    let Max = VectorFloat(1.0, 1.0, 1.0)
    let Min = VectorFloat(-1.0, -1.0, -1.0)

    let result = OctantBoundary.create Min Max

    Assert.Equal(Min, result.Min)     
    Assert.Equal(Max, result.Max) 
Ade Miller
  • 13,575
  • 1
  • 42
  • 75
  • As of 1.9.1 Xunit's new overloads seem to be causing havoc with my F#. – Rick Minerich Jul 18 '12 at 22:40
  • @RickMinerich I have experienced the same thing happening to my code. I just ended up adding explicit type annotations so that the correct overload would be chosen. However, this *does* add more noise to your code unfortunately. – Erik Schierboom Jun 02 '13 at 18:42
7

You could have a look at FSUnit - though I haven't used it yet, it might worth a try. Certainly better than using for example (native) NUnit in F#.

Ryan Clarke
  • 197
  • 2
  • 11
ShdNx
  • 3,172
  • 5
  • 40
  • 47
  • 1
    ShdNx, why would you recommend against NUnit? Don Syme's F# book shows NUnit for testing and it looks pretty similar to using NUnit in C#. The FSUnit DSL looks cool, but for people already familiar with NUnit (Mathias has "been using TDD for years") is it your experience that using NUnit with F# is more problematic than with C# or VB? – itowlson Jan 01 '10 at 20:29
  • I second itowlson comment, and question. Most definitely NUnit in F# looks pretty odd, but besides that, are you aware of specific issues that make it a good idea to use something else? – Mathias Jan 01 '10 at 20:47
  • 1
    I'd say that "looking pretty odd" is usually a compelling reason to find something better. Looking odd means hard to read, and hard to read means bugs. (I'm assuming that "looking odd" is completely different than "looking new and/or unfamiliar" - unfamiliar will become familiar, odd will stay odd.) – James Moore Jan 02 '10 at 00:13
  • 1
    To be honest (as I've mentioned in my response), I have not used FSUnit yet, but I've read that it's very painful to use NUnit in F#. Sorry if it's not so. – ShdNx Jan 02 '10 at 11:24
  • 4
    To be clear, you will still have TestFixtures and Test members if you use FsUnit. What you won't have are the standard Assert.X calls. FsUnit simply gives you a wrapper around this part of NUnit that makes it more at home in the F# language. – Ray Vernagus Jan 02 '10 at 14:40
2

Despite being a bit late to the party, I'd like to welcome Mathias to F# (better late than never ;)) and chime in that you might like my unit testing library, Expecto

Expecto has some features you might like:

  • F# syntax throughout, tests as values; write plain F# to generate tests
  • Use the built-in Expect module, or an external lib like Unquote for assertions
  • Parallel tests by default
  • Test your Hopac code or your Async code; Expecto is async throughout
  • Pluggable logging and metrics via Logary Facade; easily write adapters for build systems, or use the timing mechanism for building an InfluxDB+Grafana dashboard of your tests' execution times
  • Built in support for BenchmarkDotNet
  • Build in support for FsCheck; makes it easy to build tests with generated/random data or building invariant-models of your object's/actor's state space

--

open Expecto

let tests =
  test "A simple test" {
    let subject = "Hello World"
    Expect.equal subject "Hello World" "The strings should equal"
  }

[<EntryPoint>]
let main args =
  runTestsWithArgs defaultConfig args tests

https://github.com/haf/expecto/

Henrik
  • 9,714
  • 5
  • 53
  • 87