how to unmarshal json object if object is returning as empty string instead of empty struct

Question

I'm receiving some data as JSON, but if a object is empty, it does not return a empty struct but a empty string instead, and when unmarshaling, it returns an error.

So instead of data being {"key":{}} is {"key":""}} , it does not work even using omitempty field

Example: https://play.golang.org/p/N1iuWBxuo1C

type Store struct {
    Title string `json:"title,omitempty"`
    Item  item   `json:"item,omitempty"`
}
type item struct {
    Price float32 `json:"price,omitempty"`
    Kind  string  `json:"kind,omitempty"`
}

func main() {
    var data1 Store
    json1 := []byte(`{"title":"hello world","item":{"price":45.2,"kind":"fruit"}}`)
    if err := json.Unmarshal(json1, &data1); err != nil {
        log.Println("1, err: ", err)
        return
    }
    log.Printf("data1: %+v\n", data1)
    var data2 Store
    json2 := []byte(`{"title":"hello world","item":{}}`)
    if err := json.Unmarshal(json2, &data2); err != nil {
        log.Println("2, err: ", err)
        return
    }
    log.Printf("data2: %+v\n", data2)
    var data3 Store
    json3 := []byte(`{"title":"hello world","item":""}`)
    if err := json.Unmarshal(json3, &data3); err != nil {
        log.Println("3, err: ", err)
        return
    }
    log.Printf("data3: %+v\n", data3)
}

I talked about an approach to this in a recent talk: https://www.youtube.com/watch?v=qYSTIoEgp7g — Jonathan Hall, Apr 25 '20 at 12:07
Why you are using ""(empty string) instead for `{}` ? It can't unmarshal string to go struct. — Eklavya, Apr 25 '20 at 12:36
@Flimzy good talk but audio it's a bit anoying. I left my like :) — John Balvin Arias, Apr 25 '20 at 13:17
@Eklavya enpoint to get the data handles it that way, I can't just contact them to change it, that's the way it is — John Balvin Arias, Apr 25 '20 at 13:18
@JohnBalvinArias: Agreed. I've asked the owner of the video source if I can run it through an audio filter and re-upload. I'll see what he says :) — Jonathan Hall, Apr 25 '20 at 13:33

score 4 · Accepted Answer · answered Apr 25 '20 at 13:18

4

You can have your item type implement the json.Unmarshaler interface.

func (i *item) UnmarshalJSON(data []byte) error {
    if string(data) == `""` {
        return nil
    }

    type tmp item
    return json.Unmarshal(data, (*tmp)(i))
}

https://play.golang.org/p/1TrD57XULo9

answered Apr 25 '20 at 13:18

mkopriva

35,176
4
57
71

1

Much cleaner approach than my answer :). – Tim Apr 25 '20 at 13:25
@mkopriva every time creating type inside UnmarshalJSON is a good approach ? It may cost more right ? – Eklavya Apr 25 '20 at 13:37
@Eklavya IIRC this is handled by the compiler since everything necessary for the type declaration is static and known at compile time, just like a `const` declared inside a function is not re-declared every time the function's executed, rather it's "lifted" outside of the scope but then tied to it, in "some way", to avoid conflicts with other declared objects with the same name. But don't quote me on that as I currently have no link to no official source to prove my claim. – mkopriva Apr 25 '20 at 13:54
@mkopriva do you think it's much better to create a type and write unmarshal for this rather create new type everytime ? and Unmarshal interface may be implemented for every new type, which may cost more right ? – Eklavya Apr 25 '20 at 14:00
@Eklavya I'm not sure I understand what you're asking. As I've stated above, according to my recollection, the type `tmp` is *not* created every time, it has no reason to be, and the compiler knows that. The type `tmp` is declared once only, again, as per my recollection (I might be wrong). – mkopriva Apr 25 '20 at 14:11
@Eklavya but let's say that I'm wrong, that indeed the type *is* created every single time the function's executed, in that case your solution, while working, is a bit strange, it should rather be something like this: https://play.golang.org/p/fb8sVD1L-NB otherwise you're forcing OP to suddenly change any reference of `item` that relies on proper json unmarshaling to be updated to the temporary type. – mkopriva Apr 25 '20 at 14:11
I am using unmarshal with `item` type and then use convert it new type which is a type on item struct. And I get this way from here https://youtu.be/qYSTIoEgp7g?t=392 – Eklavya Apr 25 '20 at 14:24
@mkopriva I am not saying you are wrong, I am just curious to know about the implementation facts. – Eklavya Apr 25 '20 at 14:32
@Eklavya there's no difference in implementation between your solution and [this](https://play.golang.org/p/YC1CtC5NV5n), however your solution requires that OP changes every `item` to `itemOrEmptyString` while the other solution doesn't require that at all. You understand that in Go if you change the type of a field, for the program to compile, you now also need to change the type of any non-empty-interface typed value to which that field is assigned, right? It is completely unnecessary here and could result in a ton of work in a real life project. – mkopriva Apr 25 '20 at 14:59

score 1 · Answer 2 · answered Apr 25 '20 at 13:02

This might be a matter of taste, but "" is a string with zero length. Not an empty object. JSON uses null to describe something empty. This works:

json3 := []byte(`{"title":"hello world","item":null}`)
    if err := json.Unmarshal(json3, &data3); err != nil {
        log.Println("3, err: ", err)
        return
}

As far as the documentation goes, omitempty is an encoding option:

The "omitempty" option specifies that the field should be omitted from the encoding if the field has an empty value, defined as false, 0, a nil pointer, a nil interface value, and any empty array, slice, map, or string.

json.Unmarshal does not specify any use of the omitempty tag.

If you don't have control over the input, use an interface type, a type switch and type assertion:

type Store struct {
    Title string `json:"title,omitempty"`
    Item  item   `json:"item,omitempty"`
}
type item struct {
    Price float32 `json:"price,omitempty"`
    Kind  string  `json:"kind,omitempty"`
}

func unmarshal(js []byte) (*Store, error) {
    var data = struct { // Intermediate var for unmarshal
        Title string
        Item  interface{}
    }{}

    if err := json.Unmarshal(js, &data); err != nil {
        return nil, err
    }

    s := &Store{Title: data.Title}

    switch item := data.Item.(type) { // type switch
    case string, nil:
        return s, nil // Item remains empty
    case map[string]interface{}:
        p, ok := item["price"].(float64) // assertion
        if ok {
            s.Item.Price = float32(p)
        }

        s.Item.Kind, _ = item["kind"].(string) // _ prevents panic
        return s, nil
    default:
        return nil, errors.New("Unknown type")
    }

}

func main() {
    jsons := [][]byte{
        []byte(`{"title":"hello world","item":{"price":45.2,"kind":"fruit"}}`),
        []byte(`{"title":"hello world","item":{}}`),
        []byte(`{"title":"hello world","item":""}`),
        []byte(`{"title":"hello world","item":null}`),
    }

    for i, js := range jsons {
        data, err := unmarshal(js)
        if err != nil {
            log.Println("1, err: ", err)
            return
        }
        log.Printf("data %d: %+v\n", i, data)
    }
}

https://play.golang.org/p/Dnq1ZVfGPE7

nice try, but the problem it's the data is big so I can't just create that function for each field, your solution it's nice for short data — John Balvin Arias, Apr 25 '20 at 13:20

Eklavya · Answer 3 · 2020-04-25T13:46:03.203

0

Create a type like type ItemOrEmptyString item

And implement Unmarshal interface for it to handle your custom case.

func(ies *ItemOrEmptyString)UnmarshalJSON(d []byte) error{
    var i item
    if string(d) == `""` {
       return nil
    }
    err := json.Unmarshal(d, &i)
    *ies  = ItemOrEmptyString(i)
    return err
}

Full code here

edited Apr 25 '20 at 13:46

answered Apr 25 '20 at 13:28

Eklavya

17,618
4
28
57

you don't neet to create a nother struct to handle the unmarshal, just with the same struct you can do it – John Balvin Arias Apr 25 '20 at 13:30
I am not creating another struct I just create a type. It is not possible without create a new type I think – Eklavya Apr 25 '20 at 13:32
you are right with the first part, its not a struct but a type; but for the second part you are wrong, it's possible to do it without creating another type, just look for the accepted solution and you will see – John Balvin Arias Apr 25 '20 at 13:36
Please see clearly in accepted solution `type tmp item`, everytime unmarshal call you are creating a type accually – Eklavya Apr 25 '20 at 13:39
1

And every time unmarshal called new type will be created, and every time you are using new type's unmarshal interface may cost more. – Eklavya Apr 25 '20 at 13:44
1

@JohnBalvinArias in a compiled language like Go types are created at compile time, so you don't need to worry about `tmp` being "re created" each time `UnmarshalJSON` is executed. – mkopriva Apr 26 '20 at 03:58

how to unmarshal json object if object is returning as empty string instead of empty struct

3 Answers3

Linked

Related