"This socket is already used by another greenlet" using locust to report response times into graphite

Question

Hello I managed to get locust to work with graphite with the following code:

class MyLocust(HttpUser):
    tasks = [MyTask]
    sock = None

    def exit_handler(self):
        self.sock.shutdown(socket.SHUT_RDWR)
        self.sock.close()
 
    def hook_request_success(self, request_type, name, response_time, response_length):
        # print(response_time)
        message="%s %d %d\n" % ("client." + name.replace('.', '-'), response_time,  time.time())
        self.sock.send(message.encode())

    def hook_request_fail(self, request_type, name, response_time, exception):
        self.request_fail_stats.append([name, request_type, response_time, exception])

    def __init__(self, parent):
        super().__init__(parent)
        self.sock = socket.socket()
        self.sock.connect( ("192.168.XX.YYYY", 2003) )

        events.request_success.add_listener(self.hook_request_success)
        events.request_failure.add_listener(self.hook_request_fail)

        atexit.register(self.exit_handler)

Adn this works quiet good while I use a small amout of users in locust, around 100.

But when the users start to go up, I get the following exception:


[2021-05-31 18:40:48,430] HMG28/ERROR/root: Uncaught exception in event handler:
Traceback (most recent call last):
  File "c:\python39\lib\site-packages\gevent\_socketcommon.py", line 722, in send
    return self._sock.send(data, flags)
BlockingIOError: [WinError 10035] A non-blocking socket operation could not be completed immediately

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\python39\lib\site-packages\locust\event.py", line 40, in fire
    handler(**kwargs)
  File "C:\Users\AAA\Desktop\AAA\locust_files\locustfile.py", line 60, in hook_request_success
    self.sock.send(message.encode())
  File "c:\python39\lib\site-packages\gevent\_socketcommon.py", line 727, in send
    self._wait(self._write_event)
  File "src\\gevent\\_hub_primitives.py", line 317, in gevent._gevent_c_hub_primitives.wait_on_socket
  File "src\\gevent\\_hub_primitives.py", line 322, in gevent._gevent_c_hub_primitives.wait_on_socket
  File "src\\gevent\\_hub_primitives.py", line 297, in gevent._gevent_c_hub_primitives._primitive_wait
gevent.exceptions.ConcurrentObjectUseError: This socket is already used by another greenlet: <bound method Waiter.switch of <gevent._gevent_c_waiter.Waiter object at 0x00000222BA8FB4F0>>

Is there a way to make the socket connections wait until they can send the data?

score 0 · Answer 1 · answered Jun 01 '21 at 15:19

Searching SO for your error, I found this possibly helpful answer:

Non-Blocking error when adding timeout to python server

If you continue to have issues, you can try using a gevent compatible socket and see if that gets you the concurrency you want more easily.

"This socket is already used by another greenlet" using locust to report response times into graphite

1 Answers1