Converting map to struct

Question

I am trying to create a generic method in Go that will fill a struct using data from a map[string]interface{}. For example, the method signature and usage might look like:

func FillStruct(data map[string]interface{}, result interface{}) {
    ...
}

type MyStruct struct {
    Name string
    Age  int64
}

myData := make(map[string]interface{})
myData["Name"] = "Tony"
myData["Age"]  = 23

result := &MyStruct{}
FillStruct(myData, result)

// result now has Name set to "Tony" and Age set to 23

I know this can be done using JSON as an intermediary; is there another more efficient way of doing this?

Using JSON as an intermediary will use reflection anyway.. assuming you're going to be using the stdlib `encoding/json` package to do that intermediate step.. Can you give an example map and example struct that this method could be used on? — Simon Whitehead, Nov 04 '14 at 21:09
Yea, that is the reason I am trying to avoid JSON. Seems like there hopefully is a more efficient method that I don't know about. — tgrosinger, Nov 04 '14 at 22:01
Can you give an example use case? As in - show some pseudocode that demonstrates what this method will do? — Simon Whitehead, Nov 04 '14 at 22:02
Mmm... there might be a way with the `unsafe` package .. but I dare not try it. Other than that .. Reflection is required, as you need to be able to query the metadata associated with a type in order to place data into its properties. It would be fairly straight forward to wrap this in `json.Marshal` + `json.Decode` calls.. but that's double the reflection. — Simon Whitehead, Nov 04 '14 at 22:19
I have removed my comment about reflection. I am more interested in just doing this as efficiently as possible. If that means using reflection that is okay. — tgrosinger, Nov 04 '14 at 22:21

score 178 · Accepted Answer · edited Nov 27 '18 at 04:27

178

The simplest way would be to use https://github.com/mitchellh/mapstructure

import "github.com/mitchellh/mapstructure"

mapstructure.Decode(myData, &result)

If you want to do it yourself, you could do something like this:

http://play.golang.org/p/tN8mxT_V9h

func SetField(obj interface{}, name string, value interface{}) error {
    structValue := reflect.ValueOf(obj).Elem()
    structFieldValue := structValue.FieldByName(name)

    if !structFieldValue.IsValid() {
        return fmt.Errorf("No such field: %s in obj", name)
    }

    if !structFieldValue.CanSet() {
        return fmt.Errorf("Cannot set %s field value", name)
    }

    structFieldType := structFieldValue.Type()
    val := reflect.ValueOf(value)
    if structFieldType != val.Type() {
        return errors.New("Provided value type didn't match obj field type")
    }

    structFieldValue.Set(val)
    return nil
}

type MyStruct struct {
    Name string
    Age  int64
}

func (s *MyStruct) FillStruct(m map[string]interface{}) error {
    for k, v := range m {
        err := SetField(s, k, v)
        if err != nil {
            return err
        }
    }
    return nil
}

func main() {
    myData := make(map[string]interface{})
    myData["Name"] = "Tony"
    myData["Age"] = int64(23)

    result := &MyStruct{}
    err := result.FillStruct(myData)
    if err != nil {
        fmt.Println(err)
    }
    fmt.Println(result)
}

edited Nov 27 '18 at 04:27

byxor

5,930
4
27
44

answered Nov 04 '14 at 22:43

dave

62,300
5
72
93

4

Thank you. I am using a slightly modified version. http://play.golang.org/p/_JuMm6HMnU – tgrosinger Nov 04 '14 at 22:58
I want the FillStruct behavior on all my various structs and not have to define `func (s MyStr...) FillStruct ...` for every one. Is it possible to define FillStruct for a base struct then have all my other structs 'inherit' that behavior? In the paradigm above it's not possible since only the base struct ... in this case "MyStruct" will actually have it's fields iterated – Michael M Aug 12 '15 at 23:31
I mean you could have it work for any struct with something like this: http://play.golang.org/p/0weG38IUA9 – dave Aug 12 '15 at 23:39
Is it possible to implement tags in Mystruct? – vicTROLLA Mar 18 '16 at 19:48
Hi @dave - You have suggested to use mapstructure. There is another answer in this same discussion by jackytse (https://stackoverflow.com/a/54741880/6385674), which first marshals the map to slice and then unmarshal it to the struct. Could you please comment on 1. It seems simpler. so, is it a bad approach? or are there pitfalls?. 2. If not, is there is any performance disadvantage with using marshal and unmarshal, with smaller JSON (<1KB)? – abhishek Jan 19 '20 at 01:52
1

@abhishek there is certainly a performance penalty you'll pay for first marshaling to text and then unmarshaling. That approach is also certainly simpler. It's a tradeoff, and generally I'd go with the simpler solution. I answered with this solution because the question stated "I know this can be done using JSON as an intermediary; is there another more efficient way of doing this?". This solution will be more efficient, the JSON solution will generally be easier to implement and reason about. – dave Jan 19 '20 at 05:28

score 92 · Answer 2 · edited Jan 21 '18 at 16:48

92

Hashicorp's https://github.com/mitchellh/mapstructure library does this out of the box:

import "github.com/mitchellh/mapstructure"

mapstructure.Decode(myData, &result)

The second result parameter has to be an address of the struct.

edited Jan 21 '18 at 16:48

sh0umik

1,549
2
17
27

answered Aug 14 '16 at 05:31

yunspace

2,532
20
20

what if the map key is `user_name` and struct filed is `UserName`? – Nicholas Jela May 23 '18 at 12:42
1

@NicholasJela it can handle that with tags https://godoc.org/github.com/mitchellh/mapstructure#ex-Decode--Tags – Circuit in the wall Jun 07 '18 at 21:53
what if map kye is _id and map name is Id then it won't decode it. – Ravi Shankar Feb 22 '19 at 14:42
Didn't work for me. – Deepak Pathak Apr 14 '22 at 17:23

jackytse · Answer 3 · 2020-01-09T07:11:47.103

75

the simplest way to do that is using encoding/json package

just for example:

package main
import (
    "fmt"
    "encoding/json"
)

type MyAddress struct {
    House string
    School string
}
type Student struct {
    Id int64
    Name string
    Scores float32
    Address MyAddress
    Labels []string
}

func Test() {

    dict := make(map[string]interface{})
    dict["id"] = 201902181425       // int
    dict["name"] = "jackytse"       // string
    dict["scores"] = 123.456        // float
    dict["address"] = map[string]string{"house":"my house", "school":"my school"}   // map
    dict["labels"] = []string{"aries", "warmhearted", "frank"}      // slice

    jsonbody, err := json.Marshal(dict)
    if err != nil {
        // do error check
        fmt.Println(err)
        return
    }

    student := Student{}
    if err := json.Unmarshal(jsonbody, &student); err != nil {
        // do error check
        fmt.Println(err)
        return
    }

    fmt.Printf("%#v\n", student)
}

func main() {
    Test()
}

edited Jan 09 '20 at 07:11

answered Feb 18 '19 at 06:55

jackytse

761
5
5

1

Thanks @jackytse. This is actually the best way to do that !! map structure doesn’t often work with nested map within an interface. So it’s better consider a map string interface and handle it as a json. – Gilles Essoki Aug 13 '19 at 08:35
1

Go playground link for the above snippet: https://play.golang.org/p/JaKxETAbsnT – Junaid Jun 06 '20 at 12:49
This is the best way, why is this not the accepted answer – karthikcru Sep 03 '21 at 19:45
2

@karthikcru because this is exactly what OP said they want to avoid doing, see "I know this can be done using JSON as an intermediary; is there another more efficient way of doing this?" – Petr Dec 23 '21 at 09:10
This should be marked as the answer, best solution!!! Thanks! – Deepak Pathak Apr 14 '22 at 17:24
sadly this seems to be the best solution – quiquelhappy Jun 08 '22 at 21:02

score 16 · Answer 4 · answered Nov 04 '14 at 22:36

16

You can do it ... it may get a bit ugly and you'll be faced with some trial and error in terms of mapping types .. but heres the basic gist of it:

func FillStruct(data map[string]interface{}, result interface{}) {
    t := reflect.ValueOf(result).Elem()
    for k, v := range data {
        val := t.FieldByName(k)
        val.Set(reflect.ValueOf(v))
    }
}

Working sample: http://play.golang.org/p/PYHz63sbvL

answered Nov 04 '14 at 22:36

Simon Whitehead

63,300
9
114
138

1

This appears to panic on zero values: `reflect: call of reflect.Value.Set on zero Value` – James Taylor Apr 23 '17 at 04:17
@JamesTaylor Yes. My answer assumes you know exactly what fields you're mapping. If you're after a similar answer with more error handling (including the error you're experiencing), I would suggest Daves answer instead. – Simon Whitehead Apr 23 '17 at 05:00

score 13 · Answer 5 · edited Aug 15 '20 at 00:58

13

There are two steps:

Convert interface to JSON Byte
Convert JSON Byte to struct

Below is an example:

dbByte, _ := json.Marshal(dbContent)
_ = json.Unmarshal(dbByte, &MyStruct)

edited Aug 15 '20 at 00:58

Gabe

5,643
3
26
54

answered Aug 14 '20 at 18:52

Nick L

179
1
3

Zombo · Answer 6 · 2021-04-21T15:42:27.237

You can roundtrip it through JSON:

package main

import (
   "bytes"
   "encoding/json"
)

func transcode(in, out interface{}) {
   buf := new(bytes.Buffer)
   json.NewEncoder(buf).Encode(in)
   json.NewDecoder(buf).Decode(out)
}

Example:

package main
import "fmt"

type myStruct struct {
   Name string
   Age  int64
}

func main() {
   myData := map[string]interface{}{
      "Name": "Tony",
      "Age": 23,
   }
   var result myStruct
   transcode(myData, &result)
   fmt.Printf("%+v\n", result) // {Name:Tony Age:23}
}

score 1 · Answer 7 · answered Jul 14 '16 at 09:38

I adapt dave's answer, and add a recursive feature. I'm still working on a more user friendly version. For example, a number string in the map should be able to be converted to int in the struct.

package main

import (
    "fmt"
    "reflect"
)

func SetField(obj interface{}, name string, value interface{}) error {

    structValue := reflect.ValueOf(obj).Elem()
    fieldVal := structValue.FieldByName(name)

    if !fieldVal.IsValid() {
        return fmt.Errorf("No such field: %s in obj", name)
    }

    if !fieldVal.CanSet() {
        return fmt.Errorf("Cannot set %s field value", name)
    }

    val := reflect.ValueOf(value)

    if fieldVal.Type() != val.Type() {

        if m,ok := value.(map[string]interface{}); ok {

            // if field value is struct
            if fieldVal.Kind() == reflect.Struct {
                return FillStruct(m, fieldVal.Addr().Interface())
            }

            // if field value is a pointer to struct
            if fieldVal.Kind()==reflect.Ptr && fieldVal.Type().Elem().Kind() == reflect.Struct {
                if fieldVal.IsNil() {
                    fieldVal.Set(reflect.New(fieldVal.Type().Elem()))
                }
                // fmt.Printf("recursive: %v %v\n", m,fieldVal.Interface())
                return FillStruct(m, fieldVal.Interface())
            }

        }

        return fmt.Errorf("Provided value type didn't match obj field type")
    }

    fieldVal.Set(val)
    return nil

}

func FillStruct(m map[string]interface{}, s interface{}) error {
    for k, v := range m {
        err := SetField(s, k, v)
        if err != nil {
            return err
        }
    }
    return nil
}

type OtherStruct struct {
    Name string
    Age  int64
}


type MyStruct struct {
    Name string
    Age  int64
    OtherStruct *OtherStruct
}



func main() {
    myData := make(map[string]interface{})
    myData["Name"]        = "Tony"
    myData["Age"]         = int64(23)
    OtherStruct := make(map[string]interface{})
    myData["OtherStruct"] = OtherStruct
    OtherStruct["Name"]   = "roxma"
    OtherStruct["Age"]    = int64(23)

    result := &MyStruct{}
    err := FillStruct(myData,result)
    fmt.Println(err)
    fmt.Printf("%v %v\n",result,result.OtherStruct)
}

Oraz · Answer 8 · 2022-08-25T10:26:58.300

Here function to convert map to struct by tag. If tag not exist it will find by fieldByName.

Thanks to https://gist.github.com/lelandbatey/a5c957b537bed39d1d6fb202c3b8de06

type MyStruct struct {
    Name string `json:"name"`
    ID   int    `json:"id"`
}

myStruct := &MyStruct{}

for k, v := range mapToConvert {
        err := MapToStruct(myStruct, k, v)
        if err != nil {
            fmt.Println(err)
        }
    }
func MapToStruct(s interface{}, k string, v interface{}) error {
    var jname string
    structValue := reflect.ValueOf(s).Elem()
    fieldByTagName := func(t reflect.StructTag) (string, error) {
        if jt, ok := t.Lookup("keyname"); ok {
            return strings.Split(jt, ",")[0], nil
        }
        return "", fmt.Errorf("tag provided %s does not define a json tag", k)
    }
    fieldNames := map[string]int{}
    for i := 0; i < structValue.NumField(); i++ {
        typeField := structValue.Type().Field(i)
        tag := typeField.Tag
        if string(tag) == "" {
            jname = toMapCase(typeField.Name)
        } else {
            jname, _ = fieldByTagName(tag)
        }
        fieldNames[jname] = i
    }

    fieldNum, ok := fieldNames[k]
    if !ok {
        return fmt.Errorf("field %s does not exist within the provided item", k)
    }
    fieldVal := structValue.Field(fieldNum)
    fieldVal.Set(reflect.ValueOf(v))

    return nil
}

func toMapCase(s string) (str string) {
    runes := []rune(s)
    for j := 0; j < len(runes); j++ {
        if unicode.IsUpper(runes[j]) == true {
            if j == 0 {
                str += strings.ToLower(string(runes[j]))
            } else {
                str += "_" + strings.ToLower(string(runes[j]))
            }
        } else {
            str += strings.ToLower(string(runes[j]))
        }
    }
    return str
}

score 0 · Answer 9 · answered Jan 05 '23 at 03:53

0

Simple way just marshal it json string and then unmarshat it to struct

here is the link

answered Jan 05 '23 at 03:53

Umar

155
1
10

Converting map to struct

9 Answers9

Linked