json.Unmarshal nested object into string or []byte

Question

I'm trying to Unmarshal some json so that a nested object does not get parsed but just treated as a string or []byte.

So I want to get the following:

{
    "id"  : 15,
    "foo" : { "foo": 123, "bar": "baz" }
}

Unmarshaled into:

type Bar struct {
    ID  int64  `json:"id"`
    Foo []byte `json:"foo"`
}

I get the following error:

json: cannot unmarshal object into Go value of type []uint8

playground demo

Why not use `map[string]interface{}`? It also has the advantage of re-Marshalling the correct way. — J. Holmes, Nov 20 '13 at 17:43
@JamesHolmes this is usually not recommended since this allows for any type, if you don't explicitly want ALL types to be supported don't use empty interface (interface{}), it will cause you more problems than it will solve — Adrian Forsius, Nov 27 '18 at 15:48

ANisus · Accepted Answer · 2013-11-20T20:02:03.487

56

I think what you are looking for is the RawMessage type in the encoding/json package.

The documentation states:

type RawMessage []byte

RawMessage is a raw encoded JSON object. It implements Marshaler and Unmarshaler and can be used to delay JSON decoding or precompute a JSON encoding.

Here is a working example of using RawMessage:

package main

import (
    "encoding/json"
    "fmt"
)

var jsonStr = []byte(`{
    "id"  : 15,
    "foo" : { "foo": 123, "bar": "baz" }
}`)

type Bar struct {
    Id  int64           `json:"id"`
    Foo json.RawMessage `json:"foo"`
}

func main() {
    var bar Bar

    err := json.Unmarshal(jsonStr, &bar)
    if err != nil {
        panic(err)
    }
    fmt.Printf("%+v\n", bar)
}

Output:

{Id:15 Foo:[123 32 34 102 111 111 34 58 32 49 50 51 44 32 34 98 97 114 34 58 32 34 98 97 122 34 32 125]}

Playground

edited Nov 20 '13 at 20:02

answered Nov 20 '13 at 19:52

ANisus

74,460
29
162
158

1

that's exactly what I wanted – Ilia Choly Nov 20 '13 at 19:58
5

Told you so ;) . Glad to help! – ANisus Nov 20 '13 at 19:59
i get that output, but how do I use it? – chovy Mar 04 '21 at 16:31

score 3 · Answer 2 · edited Apr 09 '15 at 04:21

3

The Foo type is a map[string]string so define Foo correctly:

type Bar struct {
    id int64
    Foo map[string]string
}

Think that would work better

edited Apr 09 '15 at 04:21

captncraig

22,118
17
108
151

answered Nov 20 '13 at 18:53

gigatropolis

229
2
7

1

So `[]bytes` -> `Unmarshal` -> `map[string]interface{}` -> `Marshal` -> `string` ... why ? – Ilia Choly Nov 20 '13 at 19:28

score 2 · Answer 3 · answered Nov 20 '13 at 18:44

2

Defining a type which implements the Unmarshaler interface gives you access to the []byte's being parsed.

type Prefs []byte

func (p *Prefs) UnmarshalJSON(b []byte) error {
    *p = make(Prefs, len(b))
    copy(*p, b)
    return nil
}

playground demo

answered Nov 20 '13 at 18:44

Ilia Choly

18,070
14
92
160

score -1 · Answer 4 · answered Nov 20 '13 at 19:27

After a bit of tinkering I've found that in your playground demo the biggest problem is the typecasting of the json to a []byte. To see what I mean by that, take a look a this playground: http://play.golang.org/p/M0706KCZbh

If you run it, you'll notice the []byte between the typecast slice and the marshaled slice differ around the point of the 'Prefs' variable.

json marshaled from struct

[123 34 105 100 34 58 49 53 44 34 112 114 101 102 115 34 58 34 101 121 65 105 90...

typecast []byte

[123 34 105 100 34 58 49 53 44 34 112 114 101 102 115 34 58 123 34 102 111 111 34...

I've removed white space to try and make it line up as much as possible. The main takeaway from this is that typecasting does not produce the same results as running the data through the json.Marshal method and to make this work you would need a custom type to handle the unmarshaling of what the json package does not recognize.

I think this example better communicates what you're saying http://play.golang.org/p/1XFn6zqME8 . However, it doesn't really answer the question. — Ilia Choly, Nov 20 '13 at 20:02

json.Unmarshal nested object into string or []byte

4 Answers4

Linked