Python Cannot rebuild SSL socket after putting in Multiprocessing.Queue

Question

I have a server that has one process waiting for connections and then puts the sockets into a queue so worker threads can pick up a connection and communicate with the clients and do whatever.

Receiving process

    while True:
        # Wait for a connection
        print >>sys.stdout, 'waiting for a connection'

        readySocks = select.select(socksToCheck, [], [], d_consts.SELECT_SEC)
        for s in readySocks[0]:
            if s is serverClient:
                sock, client_address = s.accept()
                print >>sys.stdout, 'Client connection from ', client_address

                sockfd = multiprocessing.reduction.reduce_handle(sock.fileno())
                self.cmpQueue.put(inter_proc_msgs.ConMsg(sockfd, client_address))
                print >>sys.stdout, 'Client connection sent for processing: ', client_address

Worker process

        try:
            print '[%s] Checking Queue...' % self.name
            clientConMsg = self.cmpQueue.get(block=False) #throws exception if empty
            print 'something in queue'
            fileDescriptor = multiprocessing.reduction.rebuild_handle(clientConMsg.sockfd )
            newCon = socket.fromfd(fileDescriptor, socket.AF_INET, socket.SOCK_STREAM)
            print self.name, newCon, time.strftime("%a, %d %b %Y %H:%M:%S", time.gmtime())

            #wrap in ssl
            sslsock = ssl.wrap_socket(newCon,
                                 server_side=True,
                                 certfile="newcert.pem",
                                 keyfile="privkey.pem",
                                 ssl_version=ssl.PROTOCOL_SSLv23)
            ##non blocking
            sslsock.setblocking(0)
            #add to the list of client sockets to check with TTL
            self.clientSocketsWTTL.appendleft(self.createTupleSocknTTL(newCon, clientConMsg.clientAdd))

        except Queue.Empty:
            print '[%s] X Nothing in Queue...' % self.name 
            #pass

Now this used to work before I put SSL, but now with SSL I get:

Traceback

Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/max/workspace/Dealer/src/client_msg_processor.py", line 35, in run
    self.processClientSideTasks()
  File "/home/max/workspace/Dealer/src/client_msg_processor.py", line 66, in processClientSideTasks
    self.testSSL()
  File "/home/max/workspace/Dealer/src/client_msg_processor.py", line 113, in testSSL
    ssl_version=ssl.PROTOCOL_SSLv23)
  File "/usr/lib/python2.7/ssl.py", line 381, in wrap_socket
    ciphers=ciphers)
  File "/usr/lib/python2.7/ssl.py", line 111, in __init__
    socket.__init__(self, _sock=sock._sock)
AttributeError: '_socket.socket' object has no attribute '_sock'

Any idea if this can be fixed? I am desperately trying to avoid having to restructure the whole server.

score 1 · Accepted Answer · answered May 24 '13 at 17:27

1

Doing newCon = socket.socket(_sock=newCon) fixes the problem.

see http://www.velocityreviews.com/forums/t557014-socket-vs-_socketobject.html

answered May 24 '13 at 17:27

unixsnob

1,685
2
19
45

I'm having a similar problem and I'm trying to fix it with the same trick, but I'm getting this error: `sock = socket._realsocket(*args, _sock=sock, **kwargs) UnboundLocalError: local variable 'sock' referenced before assignment` It's a sensible error, but clearly it's not an expected one, since this was an accepted answer without further comment, and the linked forum post doesn't mention this problem either... – Alex Jan 30 '15 at 12:55
@Alex, it's been a long time since I used this fix. Are you doing this after rebuilding the socket? As far as I remember, this just refreshed some internal fields within socket. If you cannot find a solution, you will need to check socket's source code. I'm sorry I can't be of further help. – unixsnob Jan 30 '15 at 16:35
this actually got complicated enough for me that I asked [a whole new question](http://stackoverflow.com/questions/28239118/python-throttling-socket-using-ssl) to try to get it fixed. I'm not just using SSL, I'm trying to use SSL *and* throttle my connection both at once. It's not going so well, heh. – Alex Jan 30 '15 at 16:54

Python Cannot rebuild SSL socket after putting in Multiprocessing.Queue

1 Answers1