7

I always thought that good unit test, are test that are independent. By 'independent' i mean that when function 'A' uses 'B', and we test function 'A', we mock/stub out 'B' in case when 'B' does not work correctly it will not fail 'A'.

But when we check sources of golang packages that principle is not respected.

For example lets check url.go and url_test.go in url packages:

url.go:

func parseQuery(m Values, query string) (err error) {
    for query != "" {
    ...
    key, err1 := QueryUnescape(key)
    ...

url_test.go:

func TestParseQuery(t *testing.T) {
    for i, test := range parseTests {
        form, err := ParseQuery(test.query)
        ...

As we can see parseQuery use QueryUnescape. In test QueryUnescape is not stubbed in any way. So if QueryUnescape will work wrong, parseQuerytest will fail.

So authors of package not always meet requirement of 'independent' unit test. What was the reason to not bother about this principle in this case, is there some rule that allow programmer of accepting this form of unit tests?

After writing independent test in python, I'am little confused about balancing between writing perfect test(which much affects in golang about design of code) and results.

Volker
  • 40,468
  • 7
  • 81
  • 87
badray
  • 300
  • 2
  • 8
  • 8
    It is fine to have "principles" but is there any benefit from sticking to this "independent test principle" except you have to write much more code and have to debug and maintain your stubs and mocks? Where would you stop: Would you stub/mock away language constructs just because the compiler might be buggy and your tests are failing because if a buggy compiler is worth stubbing basic functionalities away? Probably not. – Volker Feb 27 '14 at 12:26
  • 1
    Yes there are a benefits, quicker test running, test are pointing me to reason of code failure, not just telling it **not** works, etc. But i don't have say that every should stick in every case with this principle, just asking where I should stop :), and i just want to know how You are dealing with this problem. – badray Feb 27 '14 at 12:33
  • 5
    How on earth should stubbing `QueryUnescape` make the test "faster" on any reasonable definition of faster? Your tests gain 200 microseconds? How often do you run your tests that this gain outweighs your work? To the claim "are pointing me to reason of code failure, not just telling it not works". So you have two packages A and B. Package B imports A. Just check A before working on B with `go test A`. Pass: Great. Now if B fails B is broken. No need for stubbing. – Volker Feb 27 '14 at 13:03
  • 2
    I didn't mean this particular case. But for example, if you have a function that use IO in case if you have hundred of tests, you should see some performance gain. Ok - but when we have such a chain of 'check requirements' sometimes we will need to check that many dependent test have run to run one test - sounds bad?. – badray Feb 27 '14 at 13:16
  • This contains some ref about independency advantages: http://stackoverflow.com/questions/61400/what-makes-a-good-unit-test also book Art of Unit Testing covers this topic. – badray Feb 27 '14 at 13:21
  • Ah, but this is something completely different (IMHO). Of course external stuff is mocked/subbed/faked. See e.g. http://golang.org/src/pkg/database/sql/fakedb_test.go which mocks kinda SQL database. – Volker Feb 27 '14 at 13:50
  • 1
    The very good answer in http://stackoverflow.com/a/61868/1266756 states independence also and in a way I can commit to: Each unit test should be independent from *any* *other* unit test (per package) and test only one functionality. But this does not disallow to use functions from other or the same package. – Volker Feb 27 '14 at 13:53
  • But there is also pointed that there 'No assumptions about order of test execution', so we cant check A before B, becouse this is 'assumpion about order of test execution'. And I didn't ever said that we can use other function from the same or other package... Just to make test independent, we must moch/stub it – badray Feb 27 '14 at 15:01
  • This A/B example was about testing different packages and has nothing to do with the valid argument that the tests inside one package should be independent. Which they *are* in the Go standard library. And no, there is really no need to mock `func f()` inside `func TestB()`, mocking such stuff is just useless and has nothing to do with "independent" tests. It is even a bad idea as now you would have to write tests for your mocks. – Volker Feb 27 '14 at 16:01
  • let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/48599/discussion-between-badray-and-volker) – badray Feb 27 '14 at 17:00

1 Answers1

5

Your question is really about defining the scope of a "unit" when "unit testing", which can be tricky. A "unit" does not necessarily mean a function, or a struct. It might mean a few functions/structs working together. But it most certainly means that you will not cross process boundaries (like using the real file system or going over a network, hitting a database, etc...). The boundaries are tested with integration tests or simulated using test doubles (mocks, stubs, spies, etc...).

I'm the author of GoConvey, a testing tool that build on top of the built-in go "testing" package. GoConvey has a comprehensive suite of unit tests. In the tests for the core of the project I don't write tests that invoke every single exported and non-exported function. I wrote the tests with the public-facing API in mind (the exported functions/structs). The implementation of that API is not the concern of my unit tests but the outcome is. You can see what I mean just by reading a few tests. I only call a few public-facing methods and assert that the result is correct. There are actually lots of structs and functions being invoked but the test coverage in that package is very high (currently 92.8%--not as high as I'd like but it's pretty good).

In the case of the convey package within GoConvey the unit is the entire package and its components. In the beginning I remember the tests being more granular, which allowed me to get into the red-green-refactor cycle more quickly. As the code grew, those smaller-scoped tests were superseded by broader tests that seemed more appropriate. So the suite will evolve with the production code. You'll feel a lot of friction and inertia if every single method is tested directly as you won't be able to change anything without breaking a bunch of tests.

Here's another interesting summary about how to discern the scope of a unit test:

http://blog.8thlight.com/uncle-bob/2014/01/27/TheChickenOrTheRoad.html

Ufuk Hacıoğulları
  • 37,978
  • 12
  • 114
  • 156
Michael Whatcott
  • 5,603
  • 6
  • 36
  • 50
  • 1
    Thanks for reply. Your answer (and link to article) was very helpful for 'defining scope of unit'. Ok so test in standard Go library are good sample of unit tests or not? For me this technique is something that other told me that this isn't a unit testing but integration testing or maybe test that were left after TDD development :). EDIT: i just read that author of art of unit testing changes his mind about 'unit' in unit testing, and that is fitting also yours answer http://artofunittesting.com/definition-of-a-unit-test/. Also there is one confused person like I about the scope :). – badray Feb 28 '14 at 12:16
  • That's a great link from Roy Osherove. Thanks for sharing! Admittedly, I haven't studied very much of the go library unit tests so I'd rather not generalize about the effectiveness of their testing strategies. – Michael Whatcott Feb 28 '14 at 18:26