Read whole message with bufio.NewReader(conn)

Question

I am working on a simple chat server and client in golang. I am having some trouble with reading messages from the net.Conn. So far this is what I have been doing:

bufio.NewReader(conn).ReadString('\n')

Since the user presses enter to send the message I only have to read until '\n'. But I am now working on encryption and when sending the public keys between client and server the key sometimes contains '\n', which makes it hard to get the whole key. I am just wondering how I can read the whole message instead of stopping at a specific character. Thanks!

TCP is a stream oriented protocol, it doesn't have "messages". You need a different protocol to frame your messages. You can simply use a length prefix, netstrings, bencode, HTTP, there are numerous options. — JimB, May 24 '17 at 13:43
@JimB Could you give me a simple example using length prefix? This is my first project using golang and I don't have much experience with networking. Thanks. — Pådne, May 24 '17 at 13:45
Aside from the framing issue, the code `bufio.NewReader(conn).ReadString('\n')` will be a source of issues because the buffer is discarded. Create the bufio.Reader once per connection, not once each time the app reads from the connection. — Charlie Tumahai, May 24 '17 at 13:55

JimB · Answer 1 · 2017-05-24T14:21:10.677

A simple option for sending binary data is to use a length prefix. Encode the data size as a 32bit big endian integer, then read that amount of data.

// create the length prefix
prefix := make([]byte, 4)
binary.BigEndian.PutUint32(prefix, uint32(len(message)))

// write the prefix and the data to the stream (checking errors)
_, err := conn.Write(prefix)
_, err = conn.Write(message)

And to read the message

// read the length prefix
prefix := make([]byte, 4)
_, err = io.ReadFull(conn, prefix)


length := binary.BigEndian.Uint32(prefix)
// verify length if there are restrictions

message = make([]byte, int(length))
_, err = io.ReadFull(conn, message)

See also Golang: TCP client/server data delimiter

You can also of course use an existing, well test protocol, like HTTP, IRC, etc. for your messaging needs. The go std library comes with a simple textproto package, or you could opt to enclose the messages in a uniform encoding, like JSON.

Read whole message with bufio.NewReader(conn)

1 Answers1