Unmarshal CSV record into struct in Go

Question

The problem how to automatically deserialize/unmarshal record from CSV file into Go struct.

For example, I have

type Test struct {
  Name string
  Surname string
  Age int
}

And CSV file contains records

John;Smith;42
Piter;Abel;50

Is there an easy way to unmarshal those records into struct except by using "encoding/csv" package for reading record and then doing something like

record, _ := reader.Read()
test := Test{record[0],record[1],atoi(record[2])}

Answer 1

There is gocarina/gocsv which handles custom struct in the same way encoding/json does. You can also write custom marshaller and unmarshaller for specific types.

Example:

type Client struct {
    Id      string `csv:"client_id"` // .csv column headers
    Name    string `csv:"client_name"`
    Age     string `csv:"client_age"`
}

func main() {
    in, err := os.Open("clients.csv")
    if err != nil {
        panic(err)
    }
    defer in.Close()

    clients := []*Client{}

    if err := gocsv.UnmarshalFile(in, &clients); err != nil {
        panic(err)
    }
    for _, client := range clients {
        fmt.Println("Hello, ", client.Name)
    }
}

Answer 2

Seems I've done with automatic marshaling of CSV records into structs (limited to string and int). Hope this would be useful.

Here is a link to playground: http://play.golang.org/p/kwc32A5mJf

func Unmarshal(reader *csv.Reader, v interface{}) error {
    record, err := reader.Read()
    if err != nil {
        return err
    }
    s := reflect.ValueOf(v).Elem()
    if s.NumField() != len(record) {
        return &FieldMismatch{s.NumField(), len(record)}
    }
    for i := 0; i < s.NumField(); i++ {
        f := s.Field(i)
        switch f.Type().String() {
        case "string":
            f.SetString(record[i])
        case "int":
            ival, err := strconv.ParseInt(record[i], 10, 0)
            if err != nil {
                return err
            }
            f.SetInt(ival)
        default:
            return &UnsupportedType{f.Type().String()}
        }
    }
    return nil
}

I'll try to create github package is someone needs this implementation.

Answer 3

You could bake your own. Perhaps something like this:

package main

import (
    "fmt"
    "strconv"
    "strings"
)

type Test struct {
    Name    string
    Surname string
    Age     int
}

func (t Test) String() string {
    return fmt.Sprintf("%s;%s;%d", t.Name, t.Surname, t.Age)
}

func (t *Test) Parse(in string) {
    tmp := strings.Split(in, ";")
    t.Name = tmp[0]
    t.Surname = tmp[1]
    t.Age, _ = strconv.Atoi(tmp[2])
}

func main() {

    john := Test{"John", "Smith", 42}
    fmt.Printf("john:%v\n", john)

    johnString := john.String()
    fmt.Printf("johnString:%s\n", johnString)

    var rebornJohn Test
    rebornJohn.Parse(johnString)
    fmt.Printf("rebornJohn:%v\n", rebornJohn)

}

Answer 4

Using csvutil it is possible to give column header see example.

In your case, this could be :

package main

import (
    "encoding/csv"
    "fmt"
    "io"
    "os"

    "github.com/jszwec/csvutil"
)

type Test struct {
    Name    string
    Surname string
    Age     int
}

func main() {
    csv_file, _ := os.Open("test.csv")
    reader := csv.NewReader(csv_file)
    reader.Comma = ';'

    userHeader, _ := csvutil.Header(Test{}, "csv")
    dec, _ := csvutil.NewDecoder(reader, userHeader...)

    var users []Test
    for {
        var u Test
        if err := dec.Decode(&u); err == io.EOF {
            break
        }
        users = append(users, u)
    }

    fmt.Println(users)
}

Answer 5

A simple way to solve the problem is to use JSON as an intermediate representation.

Once you've done this, you have a variety of tools at your disposal.

You can...

• Unmarshal directly to your type (if it's all strings)
• Unmarshal to a map[string]interface{} and then make any necessary type conversions
• Unmarshal -> convert types -> remarshal JSON -> unmarshal to your type

Here's a simple generic marshalling function that enables this flow...

pairToJSON := func(header, record []string) string {
    raw := ""
    for j, v := range record {
        if j != 0 {
            raw += ",\n"
        }
        raw += "\"" + header[j] + "\":\"" + v + "\""
    }
    raw = "{\n" + raw + "\n}"
    return raw
}

The above is compatible with the []string data produced by the standard csv library.