smallcheck what does "but n did not meet ==>" and how do I know which

Question

I wrote this property

prop_lookupsymbol = forAll $ \name table scope -> case lookupsymbol name table scope of
                                                  Just (s,_) -> property $ is_ancestor s scope
                                                  Nothing ->  forAll $ \s->is_ancestor s ==> (lookupsymbol name table s) == Nothing

and ran it with smallCheck 3 prop_lookupsymbol, the results are :

Completed 9000 tests without failure.
But 9000 did not meet ==> condition.

I know it refers to ==> call in the property but what does it mean by did not meet ? Should I worry about this ? and if yes then how do I get the tests that didn't meet the condition ?

Edit

I had a mistake where is_ancestor was lacking a second parameter, so the property is this now :

   prop_lookupsymbol = forAll $ \name table scope -> case lookupsymbol name table scope of
                                                  Just (s,_) -> property $ is_ancestor s scope
                                                  Nothing ->  forAll $ \s->is_ancestor s scope ==> (lookupsymbol name table s) == Nothing

But from 9000 there are 8340 which didn't meet the condition.

Here is explanation of the types and functions above :

SymbolTable is a type synonym for HashMap (Scope,String) Symbol (HashMap.Strict from unordered-containers package), this is simply for building a compiler :).

Symbol has variety of constructors(variable,function,type,etc) and Scopedefines in what scope the symbol is defined, we have file,class,function,method(function in class),interface.

Scope has name(file name, class name , etc) and upper scope, for a class it also has the parent scope(parent as in inheritance) and a list of interfaces scopes(which the class implements), the interface scope has a parent scope along with its upper scope.

The function is_ancestor s1 s2 returns whether s1 is an upperscope of s2 (or upper-upperscope or upper-upper-upper...) or is a parent scope(or parent-parent or parent-parent-...) or one of the interfaces(or the parent's interfaces or etc), I should mention that is_ancestor s s is always true.

Finally lookupsymbol name table scope tries to find a symbol whose name is name and scope is s where is_ancestor s scope is true, its return type is Maybe (Scope,Symbol) meaning that it returns the found symbol along with the scope the symbol is defined in(and Nothing when nothing is found).

My property states this : for any name table scope, if lookupsymbol returns Just (s,_) then s must be is_ancestor of scope, but if it returns nothing then it will return nothing for any scope that is_ancestor of scope.

I'm fairly certain it is referring to the fact that false implies anything, so if `is_parent s` is false it won't even look at the rest. This is useful information if you are using an `==>` to 'filter' values from a bigger set. Technically your tests passed, but in every case (ie 9000/9000) it did not evaluate the RHS of `==>`. Seems to imply it never tests a node with parents? But I'm not certain how this number interacts with the `forAll`. — user2407038, Sep 27 '16 at 15:00
Very generally, yes, you should worry when a large majority of your tests do not meet the `==>` condition. Either your test is structured wrong or you're not generating all the test cases you mean to. — dfeuer, Sep 27 '16 at 17:12
@dfeuer that was actually the real code, thanks you showed me a typo , I'll edit my question :) — niceman, Sep 27 '16 at 21:14
8340 out of 9000 isn't *nearly* as bad as 9000 out of 9000. In fact, you might consider that acceptable! — dfeuer, Sep 27 '16 at 22:39
FYI, the relationship you call "parent" is conventionally called "ancestor". You should probably change your terminology to avoid confusion. — dfeuer, Sep 27 '16 at 23:08

smallcheck what does "but n did not meet ==>" and how do I know which

Edit

Here is explanation of the types and functions above :

0 Answers0