37

What's the best way (completeness and performance) in Golang to serialize and deserialize a struct to string and vice versa?

for example, if I have this struct:

struct Session {
   Properties map[string]interface{}
   Permissions []int64
}

I want to store it on Redis and fetch it back. I have tried to save, int and string, it's fine, but how to store struct object?

conn := redisConnectors.Get()

// set example

_, err := conn.Do(`SETEX`, `uid_key`, EXPIRE_SEC, user_id)
_, err = conn.Do(`SETEX`, `email_key`, EXPIRE_SEC, login_email)

// get example

user_id, err := redis.Int64(conn.Do(`GET`, `uid_key`))
login_email, err := redis.String(conn.Do(`GET`, `email_key`))
Kokizzu
  • 24,974
  • 37
  • 137
  • 233
  • 3
    For reference, the [boj/redistore](https://github.com/boj/redistore/blob/master/redistore.go) library serialises session data to Redis using either `encoding/gob` or `encoding/json` and just saves the raw []byte slice to Redis: https://github.com/boj/redistore/blob/master/redistore.go#L313 – elithrar Jun 09 '15 at 03:37

2 Answers2

44

Using gob and base64 could solve the problem, for example:

import (
    "encoding/base64"
    "encoding/gob"
    "bytes"
)

type SX map[string]interface{}

// go binary encoder
func ToGOB64(m SX) string {
    b := bytes.Buffer{}
    e := gob.NewEncoder(&b)
    err := e.Encode(m)
    if err != nil { fmt.Println(`failed gob Encode`, err) }
    return base64.StdEncoding.EncodeToString(b.Bytes())
}

// go binary decoder
func FromGOB64(str string) SX {
    m := SX{}
    by, err := base64.StdEncoding.DecodeString(str)
    if err != nil { fmt.Println(`failed base64 Decode`, err); }
    b := bytes.Buffer{}
    b.Write(by)
    d := gob.NewDecoder(&b)
    err = d.Decode(&m)
    if err != nil { fmt.Println(`failed gob Decode`, err); }
    return m
}

and when you need to serialize custom struct or type (for example Session struct), just add these lines:

func init() {
    gob.Register(SX{})
    gob.Register(Session{}) 
}

if you want to use other serialization format (2020) or this benchmark (2022) for dynamic structure

Kokizzu
  • 24,974
  • 37
  • 137
  • 233
  • 12
    You shouldn't even need to base64 encode the result from gob - you can just save the `[]byte` to Redis. Faster, too. JSON encoding also benches a little faster than `encoding/gob` although it's unlikely to be a substantial/noticeable difference in most apps. – elithrar Jun 09 '15 at 05:38
  • I've tried JSON encoding before, it's the reason I asked this question, but it failed to convert `map[int]..` or decode correctly, for example: `map[string]bool` when encoded and decoded back would become `map[string]interface{}` – Kokizzu Jun 09 '15 at 07:37
  • For `map[int]...` you'll need to satisfy the `json.Marshaller` and `json.Unmarshaller` interfaces as JSON keys are strings. Your `map[string]bool` should have worked fine though. – elithrar Jun 09 '15 at 08:55
16

Serialization of a struct generally uses the encoding package. However, that will work for public fields only. If you also need to serialize private fields, see this answer as an alternative.
You have several encoding choices (binary, text, json as in this example for a struct, xml, etc.). For example, the project cupcake/rdb uses encoding/binary to implement parsing and encoding of the Redis RDB file format (a binary representation of the in-memory store). Another example is guregu/rediscache, a small library for caching data in Redis.

Gilgames
  • 481
  • 4
  • 15
VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250