Golang generic database single record to json

Question

I want to retrieve records from a database and marshall those to json. I have about 30 different tables, so I want generic functions that will work with all and any of those tables. I use xorm for database access.

I have managed to create DRY functions that retrieve the data, mostly thanks to this question & answer

This works, can marshal all records to json:

type user struct {
   Id   int64  `json:"id"`
   Name string `json:"name"`
}
// type post
// etc.

type tableRecord struct {
    PrimaryKey string
    Data       interface{}
}

var ListOfTables = map[string]tableRecord{
    "users":{"id", &[]user{}},  // type user is struct for xorm with json annotation
    //"posts":{"post_id", &[]post{}},
    // etc.. 
}

for tableName, rec := range ListOfTables {
    err := xorm.Find(rec.Data)
    if err != nil {
        log.Print(err)
    }

    out, err := json.Marshal(rec.Data)
    if err != nil {
        log.Print(err)
    }
    log.Print(string(out)) // this yields json array
}

However I struggle with ability to marshal a single record to json. I have gone around looking for ways to iterate over an interface{}, that holds a slice, found this and similar topics. Tried:

switch reflect.TypeOf(reflect.ValueOf(rec.Data).Elem().Interface()).Kind() {
case reflect.Slice:
    s := reflect.ValueOf(reflect.ValueOf(rec.Data).Elem().Interface())
    for i := 0; i < s.Len(); i++ {
        entry := s.Index(i)
        log.Printf("%v\n", entry) // prints {1 John Doe}
        // log.Print(reflect.ValueOf(entry))
        data, err := json.MarshalIndent(entry, " ", "  ")
        if err != nil {
            log.Print(err)
        }
        log.Println(string(data)) // prints {} empty
    }
}  

Of course, if I specify that rec.Data is *[]user it works, but then I would have to rewrite such code for each table, which is not what I am after.

switch t := rec.Data.(type) {
case *[]user:
    for _, entry := range *t {
        // log.Printf("loop %v", entry)
        data, err := json.MarshalIndent(entry, " ", "  ")
        if err != nil {
            log.Print(err)
        }
        log.Println(string(data)) // yields needed json for single record
    }
}

Or maybe there is a completely different, better approach how to solve such - any record of database to json.

UPDATE The problem now is, that Xorm expects the struct? I will have to read xorm possibilities and limitations.

slice := record.Slice()
log.Print(reflect.TypeOf(slice))

err = env.hxo.In(record.PrimaryKey(), insertIds).Find(slice) // or &slice
if err != nil {
    log.Print(err) // Table not found
}

// this works
var slice2 []*user
err = env.hxo.In(record.PrimaryKey(), insertIds).Find(&slice2)
if err != nil {
    log.Print(err) // 
}

Answer 1

So as I mentioned in the comment, the easiest thing to do if you want to be able to get a single element from the tableRecord.Data field would be to change the field type to what it actually is:

type tableRecord struct {
    PrimaryKey string
    Data       []interface{} // slice of whatever
}

This way, you can write something very generic:

for tbl, record := range records {
    fmt.Printf("First record from table %s\n", tbl)
    b, _ := json.MarshalIndent(record[0], " ", "  ")
    fmt.Println(string(b))
    fmt.Prinln("other records...")
    b, _ = json.MarshalIndend(record[1:], " ", "  ")
    fmt.Println(string(b))
}

What I would consider if I were you is to implement an interface in my DB types, though. Something along the lines of:

type DBType interface {
    PrimaryKey() string
    TableName() string // xorm can use this to get the table name
    Slice() []DBType // can return []user or whatever
}

So you don't really need the tableRecord type anymore, and can just use a var like this:

listOfTables := []DBType{user{}, ...}
for _, tbl := range listOfTables {
    data := tbl.Slice()
    // find data here
    fmt.Printf("First record from table %s\n", tbl.TableName())
    b, _ := json.MarshalIndent(data[0], " ", "  ")
    fmt.Println(string(b))
    fmt.Prinln("other records...")
    b, _ = json.MarshalIndend(data[1:], " ", "  ")
    fmt.Println(string(b))
}

---

So the TL;DR of what was missing from my answer/comments:

• A cast from type []user{} (or []DBTable) to []interface{} doesn't work, seeing as you can't cast all elements in a slice in a single expression. You'll have to create a second slice of type []interface{} and copy over the values like so:

  slice := userVar.Slice() data := make([]interface{}, len(slice)) for i := range slice { data[i] = slice[i] // copy over type to interface{} slice } return tableRecord{userVar.PrimaryKey(), data}

I've created a small working example of how you can use the interfaces as described above.

DEMO

To avoid too much clutter, you can change the Slice func to return a []interface{} right off the bat:

func(v T) Slice() []interface{
    return []interface{
        &T{},
    }
}

What was wrong with your implementation of Slice is that you had something like this:

func (u *user) Slice() []DBTable {
    u = &user{} // you're re-assigning the receiver, losing all state!
    return []DBTable{u}
}

The receiver is a pointer type, so any reassignments you're making is going to affect the variable on which the func was called. That's not a good idea. Just use value receivers, or, if you want to be sure that the interface is only implemented on pointer variables (a common trick, used for example by gRPC) is to implement the function like so:

func(*user) Slice() []DBTable{
    return []DBTable{&user{}}
}

A nice example of this trick can be found in generated pb.go files when using protocol buffers. The message types will have a function like this:

func(*MsgType) ProtoMessage() {}