How to override string generation for all properties in FsCheck

Question

I have a bunch of types with strings all over the place. I want to write property tests for functions operating on these types, using FsCheck. For all of them, I know that I will never get strings that are null or are non alpha-numeric. Ergo I want to restrict value generation for strings accordingly.

I tried this:

let charSet= "abc" // simplified for the example
let isValidString (s:string) = null<>s && not (s |> Seq.exists (fun c -> not (charSet |> Seq.contains c)))
let createConfig left right = {LeftLanguageName= left; RightLanguageName= right}
let generateString= Arb.generate<string> |> Gen.filter isValidString
let generateConfig= createConfig <!> generateString <*> generateString 

type Generators =
  static member String() =
    { 
        new Arbitrary<string>() with
            override x.Generator = generateString
    }
  static member LanguageConfiguration()= 
    {
        new Arbitrary<LanguageConfiguration>() with
           override x.Generator = generateConfig
    }

Arb.register<Generators>() |> ignore

However, FsCheck keeps generating values like this

Falsifiable, after 14 tests (9 shrinks) (StdGen (2144073619,296598634)):
Original:
{LeftLanguageName = " 1c\J
";
 RightLanguageName = "\026z^k";} (At least one control character has been escaped as a char code, e.g. \023)
Shrunk:
{LeftLanguageName = "
";
 RightLanguageName = "";}

So obviously I am missing something, but I got no idea what.

I looked at

How to generate null strings for FsCheck tests

and

http://blog.nikosbaxevanis.com/2015/09/25/regex-constrained-strings-with-fscheck/

but neither of them tries to override string generation globally (within scope).

score 2 · Accepted Answer · answered May 21 '19 at 21:12

I was not able to reproduce the issue. Using FSCheck 2.14.0, I tried running the following in F# Script file:

#r "C:/Temp/nuget/packages/fscheck/lib/net452/FsCheck.dll"
open FsCheck

type LanguageConfiguration = 
  { LeftLanguageName:string; RightLanguageName:string }

let charSet= "abc" // simplified for the example
let isValidString (s:string) = 
  null<>s && not (s |> Seq.exists (fun c -> not (charSet |> Seq.contains c)))
let createConfig left right = {LeftLanguageName= left; RightLanguageName= right}
let generateString= Arb.generate<string> |> Gen.filter isValidString
let generateConfig= createConfig <!> generateString <*> generateString 

type Generators =
  static member String() =
    { new Arbitrary<string>() with
            override x.Generator = generateString }
  static member LanguageConfiguration()= 
    { new Arbitrary<LanguageConfiguration>() with
           override x.Generator = generateConfig }

Arb.register<Generators>() |> ignore

Check.QuickThrowOnFailure (fun (n:LanguageConfiguration) ->
  printfn "%A" n
  true)

This runs 100 tests and the output looks like this:

{LeftLanguageName = "";
 RightLanguageName = "";}
{LeftLanguageName = "";
 RightLanguageName = "";}
{LeftLanguageName = "";
 RightLanguageName = "";}
{LeftLanguageName = "";
 RightLanguageName = "";}
{LeftLanguageName = "";
 RightLanguageName = "";}

This suggests there is another issue with your test generation (maybe just in this simplfied demo), which is that your Gen.filter isValidString call basically eliminates all interesting inputs, because they are not valid.

You could fix that by doing the string generation differently - a better approach would be to generate an int value as your length and then, in a loop, generate a character from the set of allowed characters and then concatenate those characters (which gives you only valid strings).

How to override string generation for all properties in FsCheck

1 Answers1