Create unit test for archive function fs

Question

I have the following function which is working as expected. Now I want to create unit test for it. The problem is that I'm using the file system and I am not able to figure out how to do it with some mocks or any other solution. Any idea how this can be done simply in Go? Should I really create a files and check then with unit test? I'm afraid that in some system it will work and some it breaks (winodos/ mac/linux)

This is the working function:

func Zipper(src string,artifact string,target string) error {

    zf, err := os.Create(artifact)
    if err != nil {
        return err
    }
    defer zf.Close()

    ziper := zip.NewWriter(zf)
    defer ziper.Close()

    fileInfo, err := os.Stat(src)
    if err != nil {
        return err
    }

    var bs string
    if fileInfo.IsDir(); len(target) > 0 {
        bs = target
    } else {
        bs = filepath.Base(src)
    }

    if bs != "" {
        bs += "/"
    }

    filepath.Walk(src, func(path string, info os.FileInfo, err error) error {
        if err != nil {
            return err
        }

        if info.IsDir() {
            return nil
        }

        header, err := zip.FileInfoHeader(info)
        if err != nil {
            return err
        }

        if bs != "" {
            header.Name = filepath.Join(strings.TrimPrefix(path, bs))
        }

        header.Method = zip.Deflate

        writer, err := ziper.CreateHeader(header)
        if err != nil {
            return err
        }

        file, err := os.Open(path)
        if err != nil {
            return err
        }
        defer file.Close()
        _, err = io.Copy(writer, file)
        return err
    })

    return err
}

I read the following but it not much helping in my case How to mock/abstract filesystem in go?

Which part of your function do you want to test? If you want to test the zip part you could create a separat function, which takes a `io.Reader`. Such a long function is not easy to unit test. — apxp, Aug 30 '18 at 08:19
@apxp - I want to test all :) as much possible of course, you suggest to divide the function, how ? can you please provide an example ? — , Aug 30 '18 at 08:37

Jonathan Hall · Accepted Answer · 2018-08-30T08:25:53.627

2

The simplest way to test a function that depends on the filesystem, is to add some set-up and tear-down around each test, which puts the necessary files in place before running the test, then removes them after running the test.

func TestZipper(t *testing.T) {
    // Create temporary files
    defer func() {
        // Clean up temporary files
    }()
    t.Run("group", func(t *testing.T) { // This is necessary so the above defer function doesn't run too soon
        // your actual tests
    })
}

edited Aug 30 '18 at 08:25

answered Aug 30 '18 at 08:21

Jonathan Hall

75,165
16
143
189

Thank you, so you suggest to create a zip on the before and in the test extract it and test the files which has created ? – Aug 30 '18 at 08:24
Without changing your function drastically, yes, that's the approach. You could save your "golden files" in version control, too, then you don't need to create the temporary files--you just need to clean up the output files. – Jonathan Hall Aug 30 '18 at 08:26
Thanks, and to be more specific :) , so as golden file you are suggesting to keep some "small zip" under my `testdata` folder and run the test and check the function against this `golden zip` ? – Aug 30 '18 at 08:33
@JhonD: Exactly. Although that may not always work in practice. For example, zip files often contain meta data such as a date, which changes each time you run it. – Jonathan Hall Aug 30 '18 at 08:33
Ok, thanks. so if the metadata will not be the same it will not work... – Aug 30 '18 at 08:47
Well, that depends on how you're testing for success. A simple byte-for-byte comparison of files probably won't work. – Jonathan Hall Aug 30 '18 at 08:47
OK so you suggest that better approach is to extract both and compare the files inside (maybe use golden also here) ? – Aug 30 '18 at 08:50
@JhonD: That sounds like a reasonable approach. – Jonathan Hall Aug 30 '18 at 09:07

score 2 · Answer 2 · answered Aug 30 '18 at 08:27

I know that "doesn't access the file system" is part of the definition of "unit test" to a lot of people. If your profession is not in finding and defending definitions: Forget that restriction. Filesystem access is fast and fine and the go tooling even treats folders named "testdata" special: Such folders are supposed to contain test data to be used during tests.

Go (and their users) aren't very pedantic in distinguishing "unit" from "integration" tests. Take a look at the stdlib of how to test such stuff. It is more important to write relevant tests than getting childish on fs access. (Note that a file system and a database are technically external systems, but in real life you cannot compile Go code without a file system, so isolating the test from this "external system" is ridiculous.)

Maciej Długoszek · Answer 3 · 2018-08-30T08:41:32.273

0

Well, you might look for a way to manipulate filesystem in a safe way or look for a solution somewhere else. What is the responsibility of this function? Shall it prepare a zip file or write it to filesystem?

I suggest that you should take out file creation out of this function and change the function to:

func Zipper(src string, dst io.Writer, target string) error {
    ziper := zip.NewWriter(dst)
    defer ziper.Close()

This way, for a test purpose you can provide a simple buffer, while in production use your beloved filesystem!

edited Aug 30 '18 at 08:41

answered Aug 30 '18 at 08:32

Maciej Długoszek

447
3
9

The responsibility of the function is to get source which is folder and target is where to create the zip ...sounds interesting :) can you give elaborated example please how would you change all the function – Aug 30 '18 at 08:36

Create unit test for archive function fs

3 Answers3