simpleXMLRPC server python

Question

I have an issue with the simpleXMLRPC server. I have a server in my program which can be reachable from another server. Let's call server the computer which has an instance of simpleXMLServer and client the other server.

So when the client connect to my server, the server call a function in the server. This function can sometimes take long time to execute (to 5 minutes) and then send a long message.

class Report(object):    
    __instance = None

    def __new__(self):
        if self.__instance is None:
            self.__instance = object.__new__(self)
            self.__instance.__init()
        return self.__instance

    def __init(self):
        self._lastRequestTime = 0
        self._lastRequest     = ''
        self._minInterval     = 1
        self._timeout         = 2
        self.__lock           = Utils.Threading.Lock()
        self.__lockEvent      = Utils.Threading.Lock()
        self._event           = Utils.Threading.Event()
        reportThread = threading.Thread(name   = 'reportThread',
                                        target = self.reportLoop)
        reportThread.start()

    def _reportAll(self):
        tmp = Somewhere.reportAll()
        try:
            self.__lock.acquire()
            self._lastRequest = tmp
        finally:
            self.__lock.release()

    def reportLoop(self):
        while 1:
            self._event.wait()
            self._reportAll()
            try:
                self.__lockEvent.acquire()
                self._event.clear()
            finally:
                self.__lockEvent.release()

    def reportAll(self):
        if abs(Utils.Time.time() - self._lastRequestTime) > self._minInterval or len(self._lastRequest) == 0:
            self._lastRequestTime = time.time()
        else:
            return self._lastRequest

        try:
            self.__lockEvent.acquire()
            self._event.set()    
        finally:
            self.__lockEvent.release()

        try:
            self.__lock.acquire()
            return self._lastRequest
        finally:
            self.__lock.release()

Here is how I create the server :

def startListen(self):
    logging.basicConfig(level = logging.DEBUG)
    try:
        # Create server
        self._server = SimpleXMLRPCServer(("192.168.0.57", 49007), requestHandler=RequestHandler)
        self._server.register_introspection_functions()
        self._server.register_function(Report().reportAll, 'reportAll')
        self._server.serve_forever()
    except:
        return self.restart()

To avoid a long wait in the client server (that could make freeze the client program), I created a signal in my client's server that stop the thread of the client whatever happens after 5 seconds (my client is in fact an instance the daemon gmond of ganglia). That is the code of the client:

def getMessage(self):

    def timeoutHandler(signum, frame):
        raise Exceptions.BFException('Timeout  %d s lasted' % defaultTimeout)

    self._client = None
    msg = None

    # To be sure that will not freeze, we set a signal
    signal.signal(signal.SIGALRM, timeoutHandler)
    signal.alarm(defaultTimeout)

    try:
        self._client = xmlrpclib.ServerProxy('http://192.168.0.57:49007')
        msg = self._client.reportAll() 
    except:
        print 'The connection has not been established'
        self._client = None
        msg = None
    finally:
        signal.alarm(0)
        return msg

However whatever I do, sometimes it freezes completely and I get that error :

Exception happened during processing of request from ('192.168.0.85', 55417)
Traceback (most recent call last):
  File "C:\Python26\lib\SocketServer.py", line 283, in _handle_request_noblock
    self.process_request(request, client_address)
  File "C:\Python26\lib\SocketServer.py", line 309, in process_request
    self.finish_request(request, client_address)
  File "C:\Python26\lib\SocketServer.py", line 322, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Python26\lib\SocketServer.py", line 617, in __init__
    self.handle()
  File "C:\Python26\lib\BaseHTTPServer.py", line 329, in handle
    self.handle_one_request()
  File "C:\Python26\lib\BaseHTTPServer.py", line 323, in handle_one_request
    method()
  File "C:\Python26\lib\SimpleXMLRPCServer.py", line 491, in do_POST
    self.wfile.write(response)
  File "C:\Python26\lib\socket.py", line 318, in write
    self.flush()
  File "C:\Python26\lib\socket.py", line 297, in flush
    self._sock.sendall(buffer(data, write_offset, buffer_size))
error: [Errno 10053] Une connexion établie a été abandonnée par un logiciel de votre ordinateur hôte

If I do the same in a Debian distribution, I get the same error except that I get : [Errno 32] Broken Pipe.

And whatever I do, if a call again this server I always get this error after getting a first time this error, and no response at all. I don't know how to deal with that. I have been stuck for few days on that...

Could anyone help me please ?

Thanks

score 0 · Answer 1 · answered Oct 03 '13 at 10:15

Why not run a simpleXMLRPC server on the client machine as well and implement your original server with a worker thread and a queue. Your original server would push the arguments to the queue and immediately return. The worker thread waits on the queue and executes when data is available. When it has finished it sends the data back to the client as arguments to a remote procedure call in exactly the same way as it received the call from the client.

Doing it this way means that neither the client nor the server will block.

score 0 · Answer 2 · answered Jul 19 '11 at 10:44

Basically you are killing the client's connection, so the server is reporting the connection has been terminated.

I'm not sure why you're getting no response after that, but one thing to consider is that the server is single-threaded, so while the slow operation is underway no other connections can be processed.

Also note that the signal may not be delivered to the same thread as the call to signal(), so that might cause you problems, unless python does some magic behind the scenes to ensure the correct thread is notified. Similarly, I'm not sure if throwing an exception is universally allowed, certainly, from C, you shouldn't expect to always be on the main stack.

simpleXMLRPC server python

2 Answers2