I'm Trying To Figure Out: What prevents you from "gaming" unit tests if you're doing them right?

Question

So, whenever I attempt to learn Test Driven Development something always bothers me and I figure there must be something fundamental that I am missing about how unit tests and the process of test-driven-development are supposed to be "done."

Okay, so let's say you have a simple Class, called Calculator. Let's say you have one method that you are attempting to "drive" with your tests. The method is called "Add". It is supposed to take two numbers and give you the sum of those two numbers.

Let's say the test looks like this:

[TestMethod]
        public void AddingTwoAndThreeEquals5()
        {
            var myCalulator = new Calculator();
            var result = myCalulator.Add(2, 3);

            Assert.AreEqual(5, result);
        }

Makes sense as a test, right?

Let's say this is my implementation choice:

public int Add(int a, int b)
        {
            return 5;
        }

This works. The test will pass. Hey, the implementation is even about as simple as it gets--no redundancy whatsoever. How could you refactor that? You're "Green" and good to go! :)

But that is obviously a horrible implementation choice! It will fail almost instantly in Production (just imagine the use-case where "2" and "2" are "Added" together to get "5"--the thought! gasp)! If Test-Driven-Development is just focused on getting the tests to pass as quickly as possible, what is to prevent you from "gaming" the test like this? Surely this is not the quality of code Test Driven Development is supposed to produce, but where is it expected that the code would be corrected to be more "flexible" as it would obviously need to be to function even remotely well in Production?

Do there just need to be enough tests that getting them all to pass at once while doing things like this would not be possible? Is this the sort of thing that should be caught on a code-review with a peer before check-in? Is it just up to the developer to be reasonable and know that this could not possibly pass muster in Production even if it technically makes the test pass--in other words, is the developer just to use common sense to know what is and is not in the spirit of the process even if they technically are getting the tests to go to "Green"?

This is obviously an extreme example as no one would likely check code like this in. But what about Test Driven Development is supposed to prevent "driving" code like this? Its a conceptual stumbling block that I always have when I undertake to learn Test Driven Development and I feel like there is just something fundamental that I am not getting about what Test Driven Development is and is not supposed to "do" for us.

Thank you so much for bearing with me through my question!

Cheers! :)

Answer 1

This works. The test will pass. Hey, the implementation is even about as simple as it gets--no redundancy whatsoever.

Ah - but that's not true, you do have duplication here, and you need to see it.

public int Add(int a, int b)
{
  return 5;
}

becomes

public int Add(int a, int b)
{
  return 2 + 3;
}

becomes

public int Add(int a, int b)
{
  assert a == 2;
  assert b == 3;

  return a + b;
}

Now the duplication is removed, and you can move on to the next test.

---

In practice, Beck uses "refactor" somewhat differently from Fowler's definition. In Beck's examples, if a change keeps the tests passing, it counts as a refactor even though the behavior in examples-you-aren't-testing-yet might change.

So we'd expect Beck to skip that last example, and proceed directly to:

public int Add(int a, int b)
{
  return a + b;
}

---

I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence -- Kent Beck, 2008

Now, today's picture is a bit more complicated than that. Rough timeline: Kent (re)introduces test first programming when he begins writing about Extreme Programming (XP). Test Driven Development by Example is published a few years later, and by then the ideas (a) have already been spread and (b) have a number of different interpreters, with the own interpretations, goals, agendas etc.

We've got a lot of different ideas like "Simplest Thing", and "You Aren't Gonna Need It" that are common in the same communities, and people try out applying them to the tests and refactoring of TDD. Sometimes the slogans translate well, other times... not so much?

---

what about Test Driven Development is supposed to prevent "driving" code like this?

So the first thing to recognize is that "Driven" here is pure marketing garbage. There was a time when people believed that the design followed from the tests - but that didn't really work out, and in retrospect there were some pretty clear tells that it wasn't going to work out. Michael Feathers referred to this period as the Look, Ma, no hands era.

There are some cases where the implementation of the tests gives hints that there is a problem in your design (too much irrelevant set up is a common example). But structure invariant tests simply can't "drive" a structure - all "refactorings" are reversible.

What those tests can do is alert you if a small code change you make to improve the design changes any of the behaviors you measuring. If so, you can evaluate and remedy the contradiction at once (usually by reverting the small change that introduced the fault) and move on.

(Contrast this with a "test first" approach, where the test fails until you have the entire design typed in. That same test may then still fail, and now you have to go digging for the fault you introduced. The shortened feedback loop is the big idea of TDD.)

---

Now, off the top of my head, a better test is to have two randomly generated integers and the result of Add is the result of adding the first and second integer. Suddenly the test becomes way more robust.

Test Driven Development / Test Driven Design is a design practice, not a test practice. Defending against the malfeasance of the Enterprise Developer from Hell is not a TDD goal.

Answer 2

Your cheat is actually something that is quite common while driving the design ("making the test pass, committing any crime necessary"). It works with 2 and 3 and even with a virtually infinite number of other combinations of numbers. The cheat is resolved by either getting rid of duplication as VoiceOfUnreason shows (5 is duplicated in your test and in your production code, replacing it by a+b removes this duplication) or by "triangulation" (writing more tests that expect different results). TDD is not about being "artificially stupid" or even malicious.

Answer 3

The duplication here is in the test and the production code. Notice the following two lines

Assert.AreEqual(5, result);

return 5;

To remove this duplication you need to think about making your code more generic. As @EricSchaefer rightly mentioned, to make the code more generic you need to follow the triangulation technique.