Converting interfaces with json decoding

Question

Consider the following code:

package main

import (
    "fmt"
    "encoding/json"
)


func testNoPointer() interface{} {
    return []int{}
}

func testPointer() interface{} {
    return &[]int{}
}


func main() {
    jsonString := "[1]"
    
    noPointer := testNoPointer()
    fmt.Printf("%T\n", noPointer)
    err := json.Unmarshal([]byte(jsonString), &noPointer)
    if (err == nil) {
        fmt.Printf("%T\n", noPointer)
    }

    pointer := testPointer()
    err2 := json.Unmarshal([]byte(jsonString), &pointer)
    if (err2 == nil) {
        fmt.Printf("%T\n", pointer)
    }   
}

It outputs:

[]int
[]interface {}
*[]int

Why does unmarshaling erase the type information for values? As a consequence this decoded := noPointer.([]int) will throw interface conversion: interface {} is []interface {}, not []int. Is it possible to cast it to the correct type?

I don't exactly understand what you're asking. Does this help, though? https://stackoverflow.com/q/12753805/13860 — Jonathan Hall, Nov 23 '20 at 11:03
I am asking why unmarshaling changes the type of the value from []int to []interface {}. While this is not a problem if pointers are used. — Yeezus, Nov 23 '20 at 11:20
1. "Why does unmarshaling erase the type information for values?" It doesn't. You cannot erase type information in Go. Go is statically typed. 2. Using Printf to debug the type requires a _real_ good understanding of a) how typing works in Go and b) what Printf does. Don't do that. It is going to confuse you more that it will help. 3. Your problem stems from using a pointer to a interface. 4. This looks like a XY-Problem. — Volker, Nov 23 '20 at 11:32
Can you explain what's happening then after unmarshaling? Specifically why noPointer.([]int) fails @Volker Thanks — Yeezus, Nov 23 '20 at 11:46
"Specifically why noPointer.([]int)" It fails because the dynamic value of noPointer is []interface{}`. The more interesting question would be _why_ the dynamic type of noPointer is []interface{} but that is just what json.Unmarshal does when unmarshaling a JSON array. See icza's answer for details. Protip: Never (really) use a pointer to an interface value (unless you really know what you are doing and why you really want that). Just don't. — Volker, Nov 23 '20 at 12:04

score 2 · Accepted Answer · answered Nov 23 '20 at 11:48

Your testNoPointer() and testPointer() functions return interface{} values. Understand that once you put (wrap) something in an interface value, you can no longer modify that.

So in the first case you have an interface value wrapping a slice. You can't add elements to the wrapped slice, because that requires modifying the slice header (e.g. its length field), so unmarshaling in the first case cannot use the value you pass for unmarshaling. It has to create a new value. And since you're unmarshaling into an interface{} value (type of noPointer is interface{}), the encoding/json package is free to choose whatever type it sees fit for unmarshaling, and json.Unmarshal() documents that:

To unmarshal JSON into an interface value, Unmarshal stores one of these in the interface value:
bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null

You're unmarshaling a JSON array, so it chooses []interface{}.

In your second example you are wrapping *int[] in an interface value. It is again true that you cannot modify the value wrapped in the interface, but this doesn't have to happen. The encoding/json package can just modify the pointed value, so the wrapped pointer in the interface can remain unchanged, so the encoding/json package can use the pointer, and hence can retain the element type.

Note that in the second example it's important that the wrapped pointer is not nil, because if it would be nil, the encoding/json package could not use the interface value wrapping the pointer: a nil pointer points to nowhere, so a new, pointer would have to be allocated. But then again we're at a point where encoding/json unmarshals into an interface{} value (type of pointer is interface{}), so the encoding/json package would revert to creating an []interface{} for the JSON array being unmarshaled.

You can test this when a testPointer2() returns a nil pointer wrapped:

func testPointer2() interface{} {
    var x *[]int
    return x // this will be a nil pointer
}

And testing it:

pointer2 := testPointer2()
err3 := json.Unmarshal([]byte(jsonString), &pointer2)
if err3 == nil {
    fmt.Printf("%T\n", pointer2)
}

Which outputs (try it on the Go Playground):

[]interface {}

We again "lost" the type information, because the interface value wrapping a nil pointer could not be used.

See relevant issue / discussion on github: encoding/json: clarify what happens when unmarshaling into a non-empty interface{} #26946

Converting interfaces with json decoding

1 Answers1