Unmarshaling nested JSON objects

Question

There are a few questions on the topic but none of them seem to cover my case, thus I'm creating a new one.

I have JSON like the following:

{"foo":{ "bar": "1", "baz": "2" }, "more": "text"}

Is there a way to unmarshal the nested bar property and assign it directly to a struct property without creating a nested struct?

The solution I'm adopting right now is the following:

type Foo struct {
    More String `json:"more"`
    Foo  struct {
        Bar string `json:"bar"`
        Baz string `json:"baz"`
    } `json:"foo"`
    //  FooBar  string `json:"foo.bar"`
}

This is a simplified version, please ignore the verbosity. As you can see, I'd like to be able to parse and assign the value to

//  FooBar  string `json:"foo.bar"`

I've seen people using a map, but that's not my case. I basically don't care about the content of foo (which is a large object), except for a few specific elements.

What is the correct approach in this case? I'm not looking for weird hacks, thus if this is the way to go, I'm fine with that.

Answer 1

Is there a way to unmarshal the nested bar property and assign it directly to a struct property without creating a nested struct?

No, encoding/json cannot do the trick with ">some>deep>childnode" like encoding/xml can do. Nested structs is the way to go.

Answer 2

Like what Volker mentioned, nested structs is the way to go. But if you really do not want nested structs, you can override the UnmarshalJSON func.

https://play.golang.org/p/dqn5UdqFfJt

type A struct {
    FooBar string // takes foo.bar
    FooBaz string // takes foo.baz
    More   string 
}

func (a *A) UnmarshalJSON(b []byte) error {

    var f interface{}
    json.Unmarshal(b, &f)

    m := f.(map[string]interface{})

    foomap := m["foo"]
    v := foomap.(map[string]interface{})

    a.FooBar = v["bar"].(string)
    a.FooBaz = v["baz"].(string)
    a.More = m["more"].(string)

    return nil
}

Please ignore the fact that I'm not returning a proper error. I left that out for simplicity.

UPDATE: Correctly retrieving "more" value.

Answer 3

This is an example of how to unmarshall JSON responses from the Safebrowsing v4 API sbserver proxy server: https://play.golang.org/p/4rGB5da0Lt

// this example shows how to unmarshall JSON requests from the Safebrowsing v4 sbserver
package main

import (
    "fmt"
    "log"
    "encoding/json"
)

// response from sbserver POST request
type Results struct {
    Matches []Match     
}

// nested within sbserver response
type Match struct {
    ThreatType string 
    PlatformType string 
    ThreatEntryType string 
    Threat struct {
        URL string
    }
}

func main() {
    fmt.Println("Hello, playground")

    // sample POST request
    //   curl -X POST -H 'Content-Type: application/json' 
    // -d '{"threatInfo": {"threatEntries": [{"url": "http://testsafebrowsing.appspot.com/apiv4/ANY_PLATFORM/MALWARE/URL/"}]}}' 
    // http://127.0.0.1:8080/v4/threatMatches:find

    // sample JSON response
    jsonResponse := `{"matches":[{"threatType":"MALWARE","platformType":"ANY_PLATFORM","threatEntryType":"URL","threat":{"url":"http://testsafebrowsing.appspot.com/apiv4/ANY_PLATFORM/MALWARE/URL/"}}]}`

    res := &Results{}
    err := json.Unmarshal([]byte(jsonResponse), res)
        if(err!=nil) {
            log.Fatal(err)
        }

    fmt.Printf("%v\n",res)
    fmt.Printf("\tThreat Type: %s\n",res.Matches[0].ThreatType)
    fmt.Printf("\tPlatform Type: %s\n",res.Matches[0].PlatformType)
    fmt.Printf("\tThreat Entry Type: %s\n",res.Matches[0].ThreatEntryType)
    fmt.Printf("\tURL: %s\n",res.Matches[0].Threat.URL)
}

Answer 4

Yes. With gjson all you have to do now is:

bar := gjson.Get(json, "foo.bar")

bar could be a struct property if you like. Also, no maps.

Answer 5

What about anonymous fields? I'm not sure if that will constitute a "nested struct" but it's cleaner than having a nested struct declaration. What if you want to reuse the nested element elsewhere?

type NestedElement struct{
    someNumber int `json:"number"`
    someString string `json:"string"`
}

type BaseElement struct {
    NestedElement `json:"bar"`
}

Answer 6

Assign the values of nested json to struct until you know the underlying type of json keys:-

package main

import (
    "encoding/json"
    "fmt"
)

// Object
type Object struct {
    Foo map[string]map[string]string `json:"foo"`
    More string `json:"more"`
}

func main(){
    someJSONString := []byte(`{"foo":{ "bar": "1", "baz": "2" }, "more": "text"}`)
    var obj Object
    err := json.Unmarshal(someJSONString, &obj)
    if err != nil{
        fmt.Println(err)
    }
    fmt.Println("jsonObj", obj)
}

Answer 7

I was working on something like this. But is working only with structures generated from proto. https://github.com/flowup-labs/grpc-utils

in your proto

message Msg {
  Firstname string = 1 [(gogoproto.jsontag) = "name.firstname"];
  PseudoFirstname string = 2 [(gogoproto.jsontag) = "lastname"];
  EmbedMsg = 3  [(gogoproto.nullable) = false, (gogoproto.embed) = true];
  Lastname string = 4 [(gogoproto.jsontag) = "name.lastname"];
  Inside string  = 5 [(gogoproto.jsontag) = "name.inside.a.b.c"];
}

message EmbedMsg{
   Opt1 string = 1 [(gogoproto.jsontag) = "opt1"];
}

Then your output will be

{
"lastname": "Three",
"name": {
    "firstname": "One",
    "inside": {
        "a": {
            "b": {
                "c": "goo"
            }
        }
    },
    "lastname": "Two"
},
"opt1": "var"
}

Answer 8

A little late but trying to help by giving an example of GitLab GraphQL API json response and how to use it in Golang:

suppose we do an API request to GitLab with this request:

 query{
  projects{
    count,
    nodes{
     fullPath 
    }
  }
}

Which will give us a response like this:

{
  "data": {
    "projects": {
      "count": 4,
      "nodes": [
        {
          "fullPath": "mygroup/proj1"
        },
        {
          "fullPath": "mygroup/proj2"
        },
        {
          "fullPath": "mygroup/proj3"
        },
        {
          "fullPath": "mygroup/proj4"
        }
      ]
    }
  }
}

So in our response we can analyze and match the response with Go data types:

data: as the parent of all (root node)

projects: which holds a map[string]int (count) and a []map[string]string (nodes) which is a slice

Given this, we can create structs to map the response:

For the root node data we can create this struct:

type projectData struct {
    Data struct {   // because data can consist of other nested stuff
        projects    //a struct which represents the structure of the projects
    } `json:"data"` //just mapping names
}

And the struct which holds the actual data from the projects node in our API response:

type projects struct {
    Projects struct {
        Count int                 `json:"count"`
        Nodes []map[string]string `json:"nodes"`
    } `json:"projects"`
}

And to test all of this, here is the json.Unmarshal and the usage:

func projects(resp []byte){
 var p projectData  // variable of type root node struct
 if err := json.Unmarshal(resp &p); err != nil {
        fmt.Println(err)
 }

 fmt.Println(p.Data.Projects.Count) // print out the total count of projects

 for _, v := range p.Data.Projects.Nodes { //loop over all projects and print  the fullPath field
        fmt.Println(v["fullPath"])

 }
}

Answer 9

Combining map and struct allow unmarshaling nested JSON objects where the key is dynamic. => map[string]

For example: stock.json

{
  "MU": {
    "symbol": "MU",
    "title": "micro semiconductor",
    "share": 400,
    "purchase_price": 60.5,
    "target_price": 70
  },
  "LSCC":{
    "symbol": "LSCC",
    "title": "lattice semiconductor",
    "share": 200,
    "purchase_price": 20,
    "target_price": 30
  }
}

Go application

package main

import (
    "encoding/json"
    "fmt"
    "io/ioutil"
    "log"
    "os"
)

type Stock struct {
    Symbol        string  `json:"symbol"`
    Title         string  `json:"title"`
    Share         int     `json:"share"`
    PurchasePrice float64 `json:"purchase_price"`
    TargetPrice   float64 `json:"target_price"`
}
type Account map[string]Stock

func main() {
    raw, err := ioutil.ReadFile("stock.json")
    if err != nil {
        fmt.Println(err.Error())
        os.Exit(1)
    }
    var account Account
    log.Println(account)
}

The dynamic key in the hash is handle a string, and the nested object is represented by a struct.