1

I'm confused about the number of test cases used for a boolean function. Say I'm writing a function to check whether the sale price of something is over $60 dollars. 

function checkSalePrice(price) {
  return (price > 60)
}

In my Advance Placement course, they ask the minimum # of test include boundary values. So in this case, the an example set of tests are [30, 60, 90]. This course I'm taking says to only test two values, lower and higher, eg (30, 90)

Which is correct? (I know this is pondering the depth of a cup of water, but I'd like to get a few more samples as I'm new to programming)

dmr07
  • 1,388
  • 2
  • 18
  • 37
  • And unrelated: I consider `checkSalePrice()` to be a **misleading** method name. What does it mean when this method returns *true* or maybe *false*. That the price is "checked"? Or "unchecked"? In that sense, a name like `isSalesPricesBelowThreshold()` or something alike would be much more explicit. Because that tells you A) what the method is about and B) what results to expect. Many people for example use informal standards, such as "a method named *checkWhatever* will not return a value, but throw an exception in case the condition to check isn't given". – GhostCat Sep 18 '17 at 06:28

2 Answers2

4

Kent Beck wrote

I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence (I suspect this level of confidence is high compared to industry standards, but that could just be hubris). If I don't typically make a kind of mistake (like setting the wrong variables in a constructor), I don't test for it. I do tend to make sense of test errors, so I'm extra careful when I have logic with complicated conditionals. When coding on a team, I modify my strategy to carefully test code that we, collectively, tend to get wrong.

Me? I make fence post errors. So I would absolutely want to be sure that my test suite would catch the following incorrect implementation of checkSalePrice

function checkSalePrice(price) {
    return (price >= 60)
}

If I were writing checkSalePrice using test-driven-development, then I would want to calibrate my tests by ensuring that they fail before I make them pass. Since in my programming environment a trivial boolean function returns false, my flow would look like

assert checkSalePrice(61)

This would fail, because the method by default returns false. Then I would implement

function checkSalePrice(price) {
    return true
}

Now my first check passes, so I know what this boundary case is correctly covered. I would then add a new check

assert ! checkSalePrice(60)

which would fail. Providing the corrected implementation would pass the check, and now I can confidently refactor the method as necessary.

Adding a third check here for an arbitrary value isn't going to provide additional safety when changing the code, nor is it going to make the life of the next maintainer any easier, so I would settle for two cases here.

Note that the heuristic I'm using is not related to the complexity of the returned value, but the complexity of the method

Complexity of the predicate might include covering various problems reading the input. For instance, if we were passing a collection, what cases do we want to make sure are covered? J. B. Rainsberger suggested the following mnemonic

  1. zero
  2. one
  3. many
  4. lots
  5. oops

Bruce Dawson points out that there are only 4 billion floats, so maybe you should [test them all].

Do note, though, that those extra 4 billion minus two checks aren't adding a lot of design value, so we've probably crossed from TDD into a different realm.

VoiceOfUnreason
  • 52,766
  • 5
  • 49
  • 91
  • Very nice answer, although I think there is a better answer to "test 4 billion values" than "shrugging shoulders" ... see my answer. – GhostCat Sep 18 '17 at 06:29
  • If you follow third rule of TDD (which is _You are not allowed to write any more production code than is sufficient to pass the one failing unit test_) , then one or two more test cases will be required, because for satisfying second step where `assert that checkSalePrice(60) is false` i can/will write `return price == 60 ? false : true`. – Fabio Oct 03 '17 at 18:36
0

You stumbled into on of the big problems with testing in general - how many tests are good enough?!

There are basically three ways to look at this:

  • black box testing: you do not care about the internals of your MuT (method under test). You only focus on the contract of the method. In your case: should return return true when price > 60. When you think about this for while, you would find tests 30 and 90 ... and maybe 60 as well. It is always good practice to test corner cases. So the answer would be: 3
  • white box testing: you do coverage measurements of your tests - and you strive for example to hit all paths at least once. In this case, you could go with 30 and 90 - which would be resulting in 100% coverage: So the answer here: 2
  • randomized testing, as guided by QuickCheck. This approach is very much different: you don't specify test cases at all. Instead you step back and identify rules that should hold true about your MuT. And then the framework creates random input and invokes your MuT using that - trying to find examples where the aforementioned rules break.

In your case, such a rule could be that: when checkSalePrice(a) and checkSalePrice(b) then checkSalePrice(a+b). This approach feels unusual first, but as soon as start exploring its possibilities, you can find very interesting things in it. Especially when you understand that your code can provide the required "creator" functions to the framework. That allows you to use this approach to even test much more complicated, "object oriented" stuff. It is just great to watch the framework find a flaw - and to then realize that the framework will even find the "minimum" example data required to break a rule that you specified.

GhostCat
  • 137,827
  • 25
  • 176
  • 248