REST API Update by partial json

Question

We are using Golang to implement a REST API which including CRUD, in Update service, client could send partial JSON including changed fields, and we need to handle for updating entity with these changes.

Logically, we need to get entity by Id from DB to struct, and then unmarshal payload json to another struct and update entity.

However if payload json is not fully, for example I have struct

type Customer struct {
    Id      int64 `json:"id"`
    Name    string `json:"name"`
    Age     int `json:"age"`
}

And JSON request looks like

{
  "Name": "Updated name"
}

Then the customer should be updated with new name.

That's simple example, actually it could be a nested struct and nested json, how could we handle that case with golang, or event other language like Java, .NET

Answer 1

If the Update request uses the same Customer struct then the struct fields could be pointers to differentiate between zero value and value not being set in the JSON.
Now all you need to do is merge existing struct into updated Consumer struct.
For this you can use https://github.com/imdario/mergo library in Go.

package main

import (
    "fmt"
    "github.com/imdario/mergo"
    "encoding/json"
    "os"
)

type Address struct {
    City string `json:"city"`
}

type Customer struct {
    Id      int64 `json:"id"`
    Name    string `json:"name"`
    Age     int `json:"age"`
    Address *Address `json:"address"`
}


func main() {
    old1 := &Customer{Id:1, Name:"alpha", Age:5, Address:&Address{City:"Delhi"}}

    b := []byte(`{"name": "beta"}`) //no address, age specified picks from old
    up1 := new(Customer)
    json.Unmarshal(b, up1)
    if err := mergo.Merge(up1, old1); err != nil {
        fmt.Printf("err in 1st merge: %v\n", err)
        os.Exit(1)
    }
    m1, _ := json.Marshal(up1)
    fmt.Printf("merged to: %v\n", string(m1))

    old2 := &Customer{Id:1, Name:"alpha", Age:5, Address:&Address{City:"Delhi"}}
    b2 := []byte(`{ "address": {"city": "mumbai"}}`) //address specified
    up2 := new(Customer)
    json.Unmarshal(b2, up2)
    if err := mergo.Merge(up2, old2); err != nil {
        fmt.Printf("err in 1st merge: %v\n", err)
        os.Exit(1)
    }
    m2, _ := json.Marshal(up2)
    fmt.Printf("merged to: %v\n", string(m2))
}

Answer 2

from your comments it seems you are hitting the zero-value issue a lot of users with go encounter i.e. how does one tell if the input data passed a legitimate value - or was that value zeroed by default omission.

The only way to solve this is to use pointers. So in your example, changing your data struct to this:

type Customer struct {
    Id   *int64  `json:"id"`
    Name *string `json:"name"`
    Age  *int    `json:"age"`
}

Then after unmarshaling, any uninitialized fields will have nil values e.g.

var c Customer

err := json.Unmarshal(jsonData, &c)
if err != nil {
    panic(err)
}

if c.Id != nil {
    log.Println("TODO: added SQL update parms for c.Id:", *c.Id)
}
if c.Name != nil {
    log.Println("TODO: added SQL update params for c.Name:", *c.Name)
}
if c.Age != nil {
    log.Println("TODO: added SQL update parms for c.Age:", *c.Age)
}

Note: care must be taken to ensure one does not accidentally reference any nil pointers which would trigger an instant panic.

Working playground example.

Answer 3

Another way to partial unmarshal JSON like Protocol Buffers FieldMask

• First deserialize restful api request JSON to map[string]interface{}
• Then convert map[string]interface{} to a FieldMask through fieldmask
• Finally parse map[string]interface{} into a struct through mapstructure

Here is one code sample

 type Customer struct {
     Name    string `json:"name"`
     Age     int    `json:"age"`
     Address string `json:"address"`
 }

 func (p *Customer) Update(other Customer, fm fieldmask.FieldMask) {
     if len(fm) == 0 {
         *p = other
         return
     }

     if fm.Has("name") {
         p.Name = other.Name
     }
     if fm.Has("age") {
         p.Age = other.Age
     }
     if fm.Has("address") {
         p.Address = other.Address
     }
 }

 type UpdateCustomerRequest struct {
     Customer
     FieldMask fieldmask.FieldMask `json:"-"`
 }

 func (req *UpdateCustomerRequest) UnmarshalJSON(b []byte) error {
     if err := json.Unmarshal(b, &req.FieldMask); err != nil {
         return err
     }
     return mapstructure.Decode(req.FieldMask, &req.Customer)
 }

 func PartialUpdate() {
     customer := Customer{
         Name:    "foo",
         Age:     20,
         Address: "address 1",
     }
     fmt.Printf("init: %#v\n", customer)

     blob := []byte(`{"age": 10, "address": "address 2"}`)
     req := UpdateCustomerRequest{}
     if err := json.Unmarshal(blob, &req); err != nil {
         fmt.Printf("err: %#v\n", err)
     }

     customer.Update(req.Customer, req.FieldMask)
     fmt.Printf("updated: %#v\n", customer)

 }