54

I have a struct in one package that has private fields:

package foo

type Foo struct {
    x int
    y *Foo
}

And another package (for example, a white-box testing package) needs access to them:

package bar

import "../foo"

func change_foo(f *Foo) {
    f.y = nil
}

Is there a way to declare bar to be a sort of "friend" package or any other way to be able to access foo.Foo's private members from bar, but still keep them private for all other packages (perhaps something in unsafe)?

Jonathan Hall
  • 75,165
  • 16
  • 143
  • 189
Matt
  • 21,026
  • 18
  • 63
  • 115
  • Can you not fork the existing library and expose the fields that you need to modify? (note that you should assume they're unexposed for a good reason) – elithrar Aug 01 '13 at 00:51
  • 1
    @elithrar All of it is my code. So... yes, they are unexposed for a good reason; and yes, I do need to access them. – Matt Aug 01 '13 at 13:33
  • 2
    For read-only access, see https://stackoverflow.com/q/42664837/1256452 for Go >= 1.8, where the accepted answer below (calling `y.Interface()`) no longer works. – torek Aug 09 '19 at 23:50

5 Answers5

73

There is a way to read unexported members using reflect (in Go < 1.7)

func read_foo(f *Foo) {
    v := reflect.ValueOf(*f)
    y := v.FieldByName("y")
    fmt.Println(y.Interface())
}

However, trying to use y.Set, or otherwise set the field with reflect will result in the code panicking that you're trying to set an unexported field outside the package.

In short: unexported fields should be unexported for a reason, if you need to alter them either put the thing that needs to alter it in the same package, or expose/export some safe way to alter it.

That said, in the interest of fully answering the question, you can do this (and have to do it this way in Go >= 1.7)

func change_foo(f *Foo) {
    // Since structs are organized in memory order, we can advance the pointer
    // by field size until we're at the desired member. For y, we advance by 8
    // since it's the size of an int on a 64-bit machine and the int "x" is first
    // in the representation of Foo.
    //
    // If you wanted to alter x, you wouldn't advance the pointer at all, and simply
    // would need to convert ptrTof to the type (*int)
    ptrTof := unsafe.Pointer(f)
    ptrTof = unsafe.Pointer(uintptr(ptrTof) + uintptr(8)) // Or 4, if this is 32-bit

    ptrToy := (**Foo)(ptrTof)
    *ptrToy = nil // or *ptrToy = &Foo{} or whatever you want

}

This is a really, really bad idea. It's not portable, if int ever changes in size it will fail, if you ever rearrange the order of the fields in Foo, change their types, or their sizes, or add new fields before the pre-existing ones this function will merrily change the new representation to random gibberish data without telling you. I also think it might break garbage collection for this block.

Please, if you need to alter a field from outside the package either write the functionality to change it from within the package or export it.

Edit2: Since you mention White Box testing, note that if you name a file in your directory <whatever>_test.go it won't compile unless you use go test, so if you want to do white box testing, at the top declare package <yourpackage> which will give you access to unexported fields, and if you want to do black box testing then you use package <yourpackage>_test.

If you need to white box test two packages at the same time, however, I think you may be stuck and may need to rethink your design.

Linear
  • 21,074
  • 4
  • 59
  • 70
  • 2
    Great answer, and +1000 on "[...] no qualms about changing the names of things users should be unaware of". It is unexported for a reason. – mna Aug 01 '13 at 13:20
  • 2
    Just to be clear, all these packages are my own, so if I change the name of a field, I would know about it. I had two potential uses in mind: White box testing, which your solution definitely works for, but also a parser, which converts strings to the objects in the other package, whose efficiency would benefit from bypassing the usual constructors for the structs but your solution would defeat the purpose of that. Of course, I could put the parser in the same package, but I wanted to keep the various parts of the program separate (maybe that's not very Go-like?). – Matt Aug 01 '13 at 13:48
  • 6
    White box testing is easily done by putting your *_test.go files in the same package as the one under test, so you have access to unexported fields. The Go tools correctly support this use-case and will not compile your tests with your package code except when running `go test`. – mna Aug 01 '13 at 14:27
  • @Matt -- Sure, you know you changed it, but do you remember all the places you referenced it in your tests? When I make name changes, my tests fail to compile and tell me the lines I need to change. Unsafe won't do that, at best it will quietly panic when you try to get the UnsafeAddr of the zero value, at worst you'll change the wrong field and have to spend an hour tracking down the fact that you're subtly overwriting the wrong data with gibberish. – Linear Aug 01 '13 at 20:13
  • @PuerkitoBio -- I'll add the info about white box testing since the OP specifically mentioned it. – Linear Aug 01 '13 at 20:13
  • 5
    You can also get a settable version of an (unsettable but addressable) `reflect.Value` by doing [`wv := reflect.NewAt(v.Type(), unsafe.Pointer(v.UnsafeAddr())).Elem()`](https://play.golang.org/p/5rWZGUFlZp), after which `wv.Set(newValue)` will not panic. This uses `unsafe` but should not affect GC. – cpcallen May 05 '17 at 16:23
4

I assume what you're testing is a package functionality that changes the state of that package's object, but you want to verify the internals post that change to affirm the new state is correct.

What might help would be writing Get and Set function for the private fields, so they can be accessed beyond the package scope.

package foo

type Foo struct {
    x int
    y *Foo
}

func (f *Foo) GetY() *Foo {
    return f.y
}

func (f *Foo) SetY(newY *Foo) {
    f.y = newY
}

Note that the idea of these Get and Set is to limit read and/or write access to the fields, while directly exporting them gives them read+write access automatically always. A subtle difference but worth consideration if the true goal is to only read the private fields and not operate on them (which the package internals would do in there own way)

Finally, if you're not comfortable with adding these type of wrappers for all the private fields in your package, then you can write them in a new file within that package and use build tags to ignore it in your regular builds, and include it in your test builds (wherever/however you trigger your testing).

// +build whitebox

// Get() and Set() function
go test --tags=whitebox

Regular builds ignore building test files along with them, so these wont come in your final binary. If this package is used elsewhere in entirely different ecosystem, then this file wouldn't be built still because of the build tags constraint.

sakshamsaxena
  • 107
  • 11
1

I am just starting out with C++ -> Go porting and I ran across a pair of classes that were friends with each other. I am pretty sure if they are part of the same package they are friends by default, effectively.

The upper case first letter for an identifier is bound within the package. Thus they can be in separate files so long as they are in the same directory, and will have the ability to see each other's unexported fields.

Using reflect, even if it is Go stdlib, is something you should think always carefully about. It adds a lot of runtime overhead. The solution would be basically copy&paste if the two struct types you want to be friends, they simply must be in the same folder. Otherwise you have to export them. (Personally I think the woo woo about the 'risk' of exporting sensitive data is quite overblown, although if you are writing a solitary library that has no executable, maybe there can be some sense to this since users of the library will not see these fields in the GoDoc and thus not think they can depend on their existence).

Louki Sumirniy
  • 106
  • 1
  • 1
  • 6
1

Internal fields are in principle not exported from a package, which allows the author of a package to freely modify its internals without breaking any other package. Working around this using reflect or unsafe is not a good idea.

The language does deliberately not help you achieve what you want, so you're going to have to do it yourself. The simplest one is to simply merge the two packages — this is what Go unit tests typically do.

The alternative is to create an extra accessor that is only used by the bar package:

// SetYPrivate sets the value of y.  This function is private to foo and bar,
// and should not be used by other packages.  It might go away in future releases.
func (foo *Foo) SetYPrivate(y int) {
    foo.y = y
}

An example of this technique is the runtime.MemStats function in the standard library, which returns a bunch of privates of the GC implementation.

jch
  • 5,382
  • 22
  • 41
0

There are multiple "hacks" to get there. One is using go:linkname. Another solution would be to have a public setter and inspect the stack trace.

Alok
  • 3,127
  • 2
  • 16
  • 18