3

I need to create a python middleware that will do the following:

a) Accept http get/post requests from multiple clients.

b) Modify and Dispatch these requests to a backend remote application (via socket communication). I do not have any control over this remote application.

c) Receive processed results from backend application and return these results back to the requesting clients.

Now the clients are expecting a synchronous request/response scenario. But the backend application is not returning the results synchronously. That is, some requests take much longer to process than others. Hence,

Client 1 : send http request C1 --> get response R1

Client 2 : send http request C2 --> get response R2

Client 3 : send http request C3 --> get response R3

Python middleware receives them in some order: C2, C3, C1. Dispatches them in this order to backend (as non-http messages). Backend responds with results in mixed order R1, R3, R2. Python middleware should package these responses back into http response objects and send the response back to the relevant client.

Is there any sample code to program this sort of behavior. There seem to be something like 20 different web frameworks for python and I'm confused as to which one would be best for this scenario (would prefer something as lightweight as possible ... I would consider Django too heavy ... I tried bottle, but I am not sure how to go about programming that for this scenario).

================================================

Update (based on discussions below): Requests have a request id. Responses have a response id (which should match the request id that they correspond to). There is only one socket connection between the middleware and the remote backend application. While we can maintain a {request_id : ip_address} dictionary, the issue is how to construct a HTTP response object to the correct client. I assume, threading might solve this problem where each thread maintains its own response object.

G.A.
  • 1,443
  • 2
  • 22
  • 34
  • You basically need a way to identify the backend responses so you can forward it to the correct client. If that's correct, you need to explain a bit further how the middleware <-> backend communication happens. Do you use a socket per request, or a single socket? There's any kind of information that could be used to distinguish responses? – jweyrich Feb 12 '11 at 18:11
  • Thanks for your comment. The middleware uses a single persistent socket connection to the backend. All requests from middleware are forwarded via this single socket. Clients do send a request id along with their requests. Response id should match the request id. So the question remains: How does the middleware (web server) keep track of which request id belonged to which client? I mean, is there any way for a cgi script in middleware to create a db of tuples like and once a response id matches, then send a http response to clientip:clienttcpport ? – G.A. Feb 12 '11 at 20:13

3 Answers3

4

Screw frameworks. This exactly the kind of task for asyncore. This module allows event-based network programming: given a set of sockets, it calls back given handlers when data is ready on any of them. That way, threads are not necessary just to dumbly wait for data on one socket to arrive and painfully pass it to another thread. You would have to implement the http handling yourself, but examples can be found on that. Alternatively, you could use the async feature of uwsgi, which would allow your application to be integrated with an existing webserver, but that does not integrate with asyncore by default --- though it wouldn't be hard to make it work. Depends on specific needs.

regnarg
  • 676
  • 8
  • 10
3

Quoting your comment:

The middleware uses a single persistent socket connection to the backend. All requests from middleware are forwarded via this single socket. Clients do send a request id along with their requests. Response id should match the request id. So the question remains: How does the middleware (web server) keep track of which request id belonged to which client? I mean, is there any way for a cgi script in middleware to create a db of tuples like and once a response id matches, then send a http response to clientip:clienttcpport ?

Is there any special reason for doing all this processing in a middleware? You should be able to do all this in a decorator, or somewhere else, if more appropriate.

Anyway, you need to maintain a global concurrent dictionary (extend dict and protect it using threading.Lock). Upon a new request, store the given request-id as key, and associate it to the respective client (sender). Whenever your backend responds, retrieve the client from this dictionary, and remove the entry so it doesn't accumulate forever.

UPDATE: someone already extended the dictionary for you - check this answer.

Community
  • 1
  • 1
jweyrich
  • 31,198
  • 5
  • 66
  • 97
  • Yes, the middleware abstracts away the complexities of turning client requests into a format that the backend understands (clients are sending json format, but backend has a very byte specific format with bitfields etc). So how do I associate request-id to client(sender)? I looked at the request/response objects in python - didn't see any way to get sender information. Also, didn't see if I could do some operation like responseobj = new response(ip, port) and then do response.write(replyfrombackend). I was using "bottle" to map urls to functions, but may need to look at other frameworks. – G.A. Feb 13 '11 at 15:51
  • BTW - thanks for the info on the locking dict. May need that once I have understood the mechanism of associating request-id with the sender and how to respond back to that sender. Just checked "bottle" documentation and it has a ip = request.environ.get('REMOTE_ADDR') to get ip, but no port#. The same for many other python environments (pylons). Still no idea on how to make a response object for a specific client - or if that's even possible. – G.A. Feb 13 '11 at 16:08
0

Ultimately your going from the synchronous http request-response protocol from your clients to an asynchronous queuing/messaging protocol with your backend. So you've two choices (1) either make requests wait until the backend has no outstanding work, then process one (2) write something that marries the backend responses with their associated request (using a dictionary of request or something)

One way might be to run your server in one thread while dealing with your backend in another (see... Run Python HTTPServer in Background and Continue Script Execution) or maybe look at aiohttp (https://docs.aiohttp.org/en/v0.12.0/web.html)

andrew pate
  • 3,833
  • 36
  • 28