Golang map update on http handlefunc

Question

I have a question about map in go language. I want to handle the clients (http) and save some of their information using map (key (client IP) value pair) ... http handle each http client using a new thread, so I think changing (add, delete, edit) the map data will be unsafe ... Is my action safe ?

package main

import (
    "net/http"
)

func main() {
    var clientsData map[string]string
    http.HandleFunc("/", func(writer http.ResponseWriter, request *http.Request) {
        // Is this map adding, safe or i have to use thread lock (mutex or ...) ?
        clientsData[request.RemoteAddr] = ...
    })
    http.ListenAndServe("127.0.0.10:8090", nil)
}

Following https://eli.thegreenplace.net/2019/on-concurrency-in-go-http-servers/ you need to protect map access. — mpromonet, Mar 29 '22 at 18:40

mpromonet · Answer 1 · 2022-03-29T19:19:02.463

Simplying the sample in https://eli.thegreenplace.net/2019/on-concurrency-in-go-http-servers/ allow to build a simple example that show it is not safe.

Using a simple program like :

package main

import (
    "net/http"
)

func main() {
    counters := map[string]int{}
    name := "test"
    counters[name] = 0
    http.HandleFunc("/test", func(w http.ResponseWriter, req *http.Request) {
        counters[name]++
    })

    http.ListenAndServe(":8000", nil)
}

And stimulating using :

ab -n 20000 -c 200 "127.0.0.1:8000/test"

Produce exception like :

goroutine 158 [running]:
runtime.throw({0x64d95a, 0x81faa0})
        /usr/local/go/src/runtime/panic.go:1198 +0x71 fp=0xc000384980 sp=0xc000384950 pc=0x4348f1
runtime.mapaccess2_faststr(0x697360, 0xc0003a4ba0, {0x644851, 0x4})
        /usr/local/go/src/runtime/map_faststr.go:116 +0x3d4 fp=0xc0003849e8 sp=0xc000384980 pc=0x413d34
main.main.func1({0x69bf00, 0xc0003a4b60}, 0x0)
        /home/test/gohttp/main.go:13 +0x46 fp=0xc000384a48 sp=0xc0003849e8 pc=0x5eba86
net/http.HandlerFunc.ServeHTTP(0x0, {0x69bf00, 0xc0003a4b60}, 0x0)

@mh-cbon: I tried to improve this answer, showing a simple example. — mpromonet, Mar 29 '22 at 19:15

score 0 · Accepted Answer · answered Mar 29 '22 at 18:57

If you only read, it is safe.

If you need to write, you need some way to do it “thread-safe”

The first idea is to use a sync.Mutex to protect the access to the map

However this may became a bottleneck, since you may have several requests in parallel but only one can write each time. We are talking about nanoseconds…

A second approach can be use a read/write mutex to control reading and writting. Many goroutines can read, but only one can write per time.

There are other options from package sync and sync/atomic.

There is one extra approach to consider: if you need only write into this map you can consider use a buffered channel to send a structure of key/value to be consumed in a single goroutine (responsible for store it into the map)

As you can see, this approach have many Advantages IF it makes sense to your application.

You can even use channels to safe read / write via callbacks but this is another story.

If you have a doubt, write unit tests and use the race condition detector

A buffered channel is probably better and more idiomatic. – The Fool Mar 29 '22 at 19:46 — The Fool, Mar 29 '22 at 19:46

Golang map update on http handlefunc

2 Answers2