2

Current project has me taking a struct (with annotation tags) and writing the data out as a flat file. This file is a columnar file so the positioning of the data is important. These positions and lengths are set up in my struct tags at the field level.

The issue i am having is, i am passing the pointer to my []byte result slice to my functions, but no matter what i do, the original slice is not housing the data. Here is a brief sample code that demonstrates what i am doing.

package main

import (
    "fmt"
    "strconv"
)

func writeInt(value int, fieldData *[]byte, col, length int) {
    v := fmt.Sprintf("%+0" + strconv.Itoa(length) +"d", value)
    copyData(fieldData, v, col, length)
}

func writeString(value string, fieldData *[]byte, col, length int) {
    v := fmt.Sprintf("%-" + strconv.Itoa(length) + "s", value)
    copyData(fieldData, v, col, length)
}

func copyData(fieldData *[]byte, v string, col, length int) {
    data := *fieldData
    if len(data) < col + length {
        temp := make([]byte, col + length - 1)
        copy(temp, data)
        data = temp
    }
    copy(data[col - 1:length], v)
    fieldData = &data
}

func main() {
    var results []byte

    writeInt(13, &results, 1, 3)
    writeString("TEST", &results, 4, 10)

    fmt.Print(results)

}

Expected result (as string) should be:

'013TEST      ' - zero pad in front of int and space pad behind string

But i am getting []

Am i looking at this entirely wrong, or am i just not understanding something?

Jason Murphy
  • 115
  • 8
  • `fieldData = &data` does the opposite of what you want: it overwrites the local variable `fieldData` with a pointer to the local variable `data`. What you want is more likely `*fieldData = data`, overwriting *the memory pointed to* by `fieldData` with the data in `data`. – Adrian Apr 15 '22 at 16:53

2 Answers2

1

Note beforehand: Do not use pointers to slices (slices are already small headers pointing to a backing array). You may modify the elements without a pointer, and if you need to modify the header (e.g. append elements to it), return the new slice, just like the builtin append() does.

Also what you try to do is much easier achievable with the bytes.Buffer type. It implements io.Writer, you can directly write into it (even using fmt.Fprint()), and obtain its content either as a []byte or as a string.

Every parameter is a copy of the passed value. Modifying parameters only modifies this copy.

If you do:

fieldData = &data

Even though fieldData is a pointer, you're just modifying the copy. You must modify the pointed value:

*fieldData = data

Printing the results:

fmt.Println(results)
fmt.Printf("%q\n", string(results))

Output (try it on the Go Playground):

[43 49 51 84 69 83 84 32 32 32 0 0 0]
"+13TEST   \x00\x00\x00"
icza
  • 389,944
  • 63
  • 907
  • 827
0

See icza's answer, and particularly the "note beforehand" section, for your specific case. For a general discussion of pointers, read on; note that only some of this is specific to Go itself.

What a pointer does for you is to give you a level of indirection. Some languages—Go included—use a "pass by value" mechanism, where regardless of whether you pass a constant value or a variable to some function:

f(3)

or:

f(x)

the function f receives the value, not the name of the variable or anything remotely like that. (Other languages are different, having "pass by name",1 "pass by reference", or "value-result" semantics in some cases. See also Why [do] so many languages [use pass] by value? The fact that f receives the value, not the variable's name or anything like that, is helpful when there isn't a variable, as in the f(3) case, or for:

f(x + y)

where we have to do a summation first and hence there's no single variable involved.

Now, in Go in particular, functions can and very often do have multiple return values:

func g(a int) (bool, int, error) { ... }

so if we want to be able to update something, we can just write:

ok, x, err = g(x)

which gathers all three values, including the updated x, right where we want it. But this does expose the details of the updated x, and/or maybe it is inconvenient. If we want to give some function permission to change some value(s) stored in some variable, we can define our function to take a pointer to that variable:

func g2(a *int) (bool, error) { ... }

Now instead of ok, x, err = g(x) we can write ok, err = g2(&x).

This is not much of an improvement at all, for this particular case. But suppose instead of a simple int, x is now a structure with a bunch of complicated state, e.g., something that will read input from a series of files, switching to the next file automatically:

x := multiFileReader(...)

Now if we want parts of the multi-file-reader to be able to access various fields in whatever structure x represents, we can make x itself a pointer variable, pointing to the struct. Then:

str, err := readNextString(x)

passes a pointer (because x is a pointer) which allows readNextString to update some of the fields within x. (This same general logic applies if x is a method, e.g, an io.Reader, but in this case we start using interface, which adds a bunch of additional wrinkles. I'm ignoring those here to concentrate on the pointer aspect.)

Adding indirection adds complexity

When we do this sort of thing—pass a pointer value pointing to the original variable, where the original variable itself holds some initial or intermediate value and we update it as we go along—the function that receives this pointer value now has a variable of type pointer to T for some type T. This extra variable is a variable. That means we can assign a new value into it. If and when we do so, we lose the original value:

func g2(a *int) (bool, error) {
    ... section 1: do stuff with `*a` ...
    var another int
    a = &another
    ... section 2: do more stuff with `*a` ...
    return ok, err
}

In section 1, a points to, and *a thus refers to, whatever variable some caller passed to g2. Making changes to *a will show up there. But in section 2, a points to another, and *a is initially zero (because another is zero). Making changes to *a here won't show up in the caller.

You can, if you like, avoid using *a directly:

func g2(a *int) (bool, error) {
    b := *a
    p := &b
    var ok bool
    // presumably there's a `var err error` or equivalent too
    for attempts := 0; !ok && attempts < 5; attempts++ {
        ... do things ...
        ... if things are working well, set ok = true and set p = a ...
        ... update *p ...
    }
    return ok, err
}

Sometimes this is the sort of thing you want: if things aren't going well we carefully don't overwrite *a by overwriting b instead, but if things are going well we overwrite *a by having p point to a in the "update *p" section. But it's tougher to reason about: make sure you're getting a very clear benefit before you resort to this sort of thing.

And of course, if we have a pointer to some variable stored in a variable, we can take a pointer to that variable too: i := 3; a := &i; pa := &a. That gives us the opportunity to yet add another level of indirection, ppa := &pa, which gives us another opportunity, and so on. It's turtles all the way down pointers all the way up, except at the very end where we must have some sort of final answer like i.

1Pass-by-name is particularly tricky, and not very common; see What is "pass-by-name" and how does it work exactly? But it did lead to a great joke that Niklaus Wirth used to tell about himself, that some would say "Nick-louse Veert" and hence call him by name, and others would say "Nickle's Worth" and hence call him by value. (I think I heard this one second hand—I don't think he ever came to the U when I was there.)

torek
  • 448,244
  • 59
  • 642
  • 775