44

I would like to be able to log every write/read that my go app issues to the underlying OS, and also (if it's possible) completely replace FS with one that resides only in memory.

Is it possible? How? Maybe there is a ready-to-Go solution?

Jonathan Hall
  • 75,165
  • 16
  • 143
  • 189
canni
  • 5,737
  • 9
  • 46
  • 68

4 Answers4

47

This is straight from Andrew Gerrand's 10 things you (probably) don't know about Go:

var fs fileSystem = osFS{}

type fileSystem interface {
    Open(name string) (file, error)
    Stat(name string) (os.FileInfo, error)
}

type file interface {
    io.Closer
    io.Reader
    io.ReaderAt
    io.Seeker
    Stat() (os.FileInfo, error)
}

// osFS implements fileSystem using the local disk.
type osFS struct{}

func (osFS) Open(name string) (file, error)        { return os.Open(name) }
func (osFS) Stat(name string) (os.FileInfo, error) { return os.Stat(name) }

For this to work, you will need to write your code to take a fileSystem argument (maybe embed it in some other type, or let nil denote the default filesystem).

Fred Foo
  • 355,277
  • 75
  • 744
  • 836
  • 2
    That looks promising, but what about direct calls to `os.Open`? is assigning `fs` variable will "silently" affect them to? – canni May 24 '13 at 19:56
  • 1
    @canni: no, it will not affect direct calls. If that's what you want, I think you're going to have to play tricks on the linker, and I've never tried that myself. – Fred Foo May 24 '13 at 20:21
  • 1
    This implementation is really clever, but all the "files" are nil which makes it hard to do some types of testing. I.E. if you have a file as part of a struct or a map of files it is impossible to test that the correct file is mapped to the correct struct or map key. – Dave Rawks Feb 10 '15 at 23:44
  • @Paradiesstaub Here you go: [Example code for testing the filesystem in Golang](http://stackoverflow.com/questions/43912124/example-code-for-testing-the-filesystem-in-golang/43914455#43914455) – icza May 11 '17 at 11:54
33

For those looking to solve the problem of mocking out your filesystem during testing, checkout @spf13's Afero library, https://github.com/spf13/afero. It does everything that the accepted answer does, but with better documentation and examples.

Ryan Walls
  • 6,962
  • 1
  • 39
  • 42
  • The afero library's memory based file system (what would be used for testing) is very lenient. Some functions return an error, but that error will always be nil. – Bren May 31 '18 at 15:26
  • 2
    @Bren we use Afero for our production code, not just for testing. Could you explain more what you mean about the errors being nil? Probably best to file an issue on the github repo if you think the behavior is incorrect. – Ryan Walls Jun 05 '18 at 13:26
  • 1
    It doesn't seem to be a drop-in replacement for `io/fs` though. https://github.com/spf13/afero/issues/325 – G. Kashtanov Dec 24 '21 at 13:05
  • 1
    @G.Kashtanov "You have to convert `afero.Fs` to an `afero.IOFS`" according to a reply to that issue: https://github.com/spf13/afero/issues/325#issuecomment-1115075828 – Benjamin R May 25 '23 at 05:50
21

You can use the testing/fstest package:

package main
import "testing/fstest"

func main() {
   fs := fstest.MapFS{
      "hello.txt": {
         Data: []byte("hello, world"),
      },
   }
   data, err := fs.ReadFile("hello.txt")
   if err != nil {
      panic(err)
   }
   println(string(data) == "hello, world")
}

https://godocs.io/testing/fstest

Zombo
  • 1
  • 62
  • 391
  • 407
  • 3
    A directory can be added like this: `fstest.MapFS{"tmp": {Mode: fs.ModeDir}}`. – G. Kashtanov Dec 24 '21 at 13:18
  • 2
    Note that all subdirectory passed in for tests MUST NOT be absolute paths. For example this is incorrect: `fstest.MapFS{"/do/not/use/absolute/paths/no.txt": {}}` This is correct: `fstest.MapFS{"relative/path/with/no/leading/slash/ok.txt": {}}` I'd accidentially added an absolute path and these entries got omitted within the test. Weirdly enough a directory with an empty ´name` was added instead. – Bouncing Bit Oct 28 '22 at 21:49
  • 3
    How do you support testing absolute paths? – Steve Nov 10 '22 at 23:14
1

Just because this question pops up pretty high when googling for this matter:

I don't know about logging reads and writes, but for a filesystem residing only in memory, I've found blang/vfs. I haven't used in production, and it says it's alpha and interfaces are likely to change. Use it at your own risk.

I suppose you could implement it to log reads and writes.

FrontierPsycho
  • 743
  • 7
  • 25