2

I'm just having a bit of a go with golang, I built a little load tester to play with concurrency stuff in go which looks cool.

My tester app starts a bunch of threads and each thread does a bunch of http requests as fast as possible to some endpoint. After a few seconds I get the following error:

connectex: Only one usage of each socket address (protocol/network address/port) is normally permitted.

I've come across this problem before in dotnet, its to do with running out of ports allocated by windows (https://aspnetmonsters.com/2016/08/2016-08-27-httpclientwrong/). HttpClient (dotnet) solves this by reusing ports between requests.

Does go have a similar way to reuse the OS allocated port between http requests?

Code to reproduce: 

package main

import (
    "fmt"
    "io/ioutil"
    "net/http"
    "time"
)

const numberOfCallsPerThread = 100
const numberOfThreads = 128

func main() {
    var chans [numberOfThreads]chan time.Duration
    for i := range chans {
        chans[i] = make(chan time.Duration, 5) //only let one thread get 5 calls ahead
        go callGet(numberOfCallsPerThread, chans[i])
    }

    var durations [numberOfCallsPerThread * numberOfThreads]time.Duration

    for i := 0; i < numberOfCallsPerThread; i++ {
        for j := range chans {
            durations[(i*numberOfThreads)+j] = <-chans[j]
        }
    }
}

func callGet(numberOfTimes int, responseTime chan time.Duration) {

    for i := 0; i < numberOfTimes; i++ {
        start := time.Now()
        response, httpError := http.Get("http://google.com")
        if httpError != nil {
            close(responseTime)
            panic(httpError)
        }
        responseBody, _ := ioutil.ReadAll(response.Body)
        response.Body.Close()
        elapsed := time.Since(start)
        fmt.Println(len(responseBody), "    in ", elapsed)

        responseTime <- elapsed
    }
}

(sorry no playground link as it doesn't support http things)

NOTE: This issue only occurs on windows (osx is not reproducable)

undefined
  • 33,537
  • 22
  • 129
  • 198
  • You're calling defer in a loop. None of those will execute until the function returns. – JimB Jun 27 '17 at 00:26
  • @JimB what should I be doing instead? – undefined Jun 27 '17 at 00:28
  • @jimb I tried removing the defer (given its after the read anyway) but same issue. – undefined Jun 27 '17 at 00:53
  • Yes, if you need to clean up within a loop you can't really use defer. Now since you're only connecting to one hostname, you need to set the max number of idle connections to match your concurrency. See https://stackoverflow.com/questions/39813587/go-client-program-generates-a-lot-a-sockets-in-time-wait-state/39834253#39834253 – JimB Jun 27 '17 at 02:38

0 Answers0