Working with raw bytes from a network in go

Question

(Sorry, long question!) I've recently been trying out Go as opposed to C++ for a game server emulator I'm working on as a side project and am questioning whether I'm implementing it in sensible Go terms. As you might expect, the server communicates with one or more game clients by sending raw packets (TCP) that adhere to a particular protocol specification. The relevant part goes something like this:

receive a header -> decrypt it -> recv bytes until the header length is reached -> decrypt the rest of the packet -> dispatch to a handler -> decode the packet -> handle as necessary -> send a response

The protocol is defined in terms of bytes in little endian order, so in my C++ implementation the packet header looks like this (I know, it only works on LE machines):

struct pkt_header {
    uint16_t length;
    uint16_t type;
    uint32_t flags;
};

Upon recv()'ing and decrypting this header, I'll extract the fields:

// client->recv_buffer is of type u_char[1024]
header = (pkt_header*) client->recv_buffer;

if (client->recv_size < header->length) {
    // Recv some more
}
// Decrypt and so on

In the handlers themselves I can nest the above header struct in other packet structure definitions and cast those onto the byte[] buffer arrays in order to directly access the fields. From what I've read, struct alignment (unsurprisingly) is difficult/impossible and highly discouraged in Go.

Not knowing what else to do, I wrote this function for going from an arbitrary Struct -> []byte:

// Serializes the fields of a struct to an array of bytes in the order in which the fields are
// declared. Calls panic() if data is not a struct or pointer to struct.
func StructToBytes(data interface{}) []byte {
    val := reflect.ValueOf(data)
    valKind := val.Kind()
    if valKind == reflect.Ptr {
        val = reflect.ValueOf(data).Elem()
        valKind = val.Kind()
    }

    if valKind != reflect.Struct {
        panic("data must of type struct or struct ptr, got: " + valKind.String())
    }

    bytes := new(bytes.Buffer)
    for i := 0; i < val.NumField(); i++ {
        field := val.Field(i)

        switch kind := field.Kind(); kind {
        case reflect.Struct:
            binary.Write(bytes, binary.LittleEndian, StructToBytes(field.Interface()))
        case reflect.Array, reflect.Slice:
            binary.Write(bytes, binary.LittleEndian, field.Interface())
        case reflect.Uint8:
            binary.Write(bytes, binary.LittleEndian, uint8(field.Uint()))
        case reflect.Uint16:
            binary.Write(bytes, binary.LittleEndian, uint16(field.Uint()))
        // You get the idea
        }
    }
    return bytes.Bytes()
}

And would do this in a handler:

type Header struct {
    length uint16
    size uint16
    flags uint32
}
newHeader := new(Header)
// Initialization, etc
client.Conn.Write(StructToBytes(newHeader)) // ex. [C8 00 03 00 00 00 01 00]  

As a Go newbie, feedback on how I might implement this more efficiently is more than welcome. So far it's worked well, but now I'm faced with the challenge of how to do the opposite: go from []byte->Struct (for example, [C8 00 03 00 00 01 00 00] to a Header { length = C8, size = 03, flags = 0100 }

Do I need to just implement the reverse of this or is there a better method of going from an array of bytes to an arbitrary struct (or vice versa, as opposed to my function)? Please let me know if any additional clarity would be helpful.

Answer 1

The go way would be to use encoding/binary which internally does pretty much what you've written above.

(playground)

package main

import (
    "bytes"
    "encoding/binary"
    "fmt"
    "log"
)

type Header struct {
    Length uint16
    Size   uint16
    Flags  uint32
}

func main() {
    header := &Header{Length: 0xC8, Size: 3, Flags: 0x100}
    fmt.Printf("in = %#v\n", header)
    buf := new(bytes.Buffer)

    err := binary.Write(buf, binary.LittleEndian, header)
    if err != nil {
        log.Fatalf("binary.Write failed: %v", err)
    }
    b := buf.Bytes()
    fmt.Printf("wire = % x\n", b)

    var header2 Header
    buf2 := bytes.NewReader(b)
    err = binary.Read(buf2, binary.LittleEndian, &header2)
    if err != nil {
        log.Fatalf("binary.Read failed: %v", err)
    }
    fmt.Printf("out = %#v\n", header2)
}

Which prints

in = &main.Header{Length:0xc8, Size:0x3, Flags:0x100}
wire = c8 00 03 00 00 01 00 00
out = main.Header{Length:0xc8, Size:0x3, Flags:0x100}