3

I developed till now with different webapp-servers (Tornado, Django, ...) and am encountering the same problem again and again:

I want a simple web proxy (reverse proxy) that allows me, that I can combine different source entities from other web-servers (that could be static files, dynamic content from an app server or other content) to one set of served files. That means, the browser should see them as if they come from one source.

I know, that I can do that with nginx, but I am searching for an even simpler tool for development. I want something, that can be started on command line and does not need to run as root. Changing the configuration (routing of the requests) should be as simple as possible.

In development, I just want to be able to mashup different sources. For example: On my production server runs something, that I don't want to copy, but I want to connect with static files on a different server and also a new application on my development system.

Speed of the proxy is not the issue, just flexibility and speed of development!

Preferred would be a Python or other scripting solution. I found also a big list of Python proxys, but after scanning the list, I found that all are lacking. Most of them just connect to one destination server and no way to have multiple servers (the proxy has to decide which to take by analysis of the local url).

I am just wondering, that nobody else has this need ...

Dayo
  • 12,413
  • 5
  • 52
  • 67
Juergen
  • 12,378
  • 7
  • 39
  • 55
  • Are you serving through wsgi, then i think i have an answer for you.. – brunsgaard Mar 05 '15 at 02:38
  • Some files yes, others not, that is the problem. For example Django has some odd limits on where static files can be served from. Also there are scenarios, where I want to bring different data sources together for example for debugging or quick tests. So not all. – Juergen Mar 05 '15 at 02:44
  • >"I am just wondering, that nobody else has this need ..." - yes, mirroring several sites as a development solution is rather unusual. You either develop a single site or deploy a permanent proxy to several sites. The main reason is that a site's framework needs to know its relative path to generate links on a page correctly so you can't just change proxy-site pairings on a whim. – ivan_pozdeev Mar 05 '15 at 22:17

3 Answers3

2

You do not need to start nginx as root as long as you do not let it serve on port 80. If you want it to run on port 80 as a normal user, use setcap. In combination with a script that converts between an nginx configuration file and a route specification for your reverse proxy, this should give you the most reliable solution.

If you want something simpler/smaller, it should be pretty straight-forward to write a script using Python's BaseHTTPServer and urllib. Here's an example that only implements GET, you'd have to extend it at least to POST and add some exception handling:

#!/usr/bin/env python
# encoding: utf-8

import BaseHTTPServer
import SocketServer
import urllib
import re

FORWARD_LIST = {
    '/google/(.*)': r'http://www.google.com/%s',
    '/so/(.*)': r'http://www.stackoverflow.com/%s',
}

class HTTPServer(BaseHTTPServer.HTTPServer, SocketServer.ThreadingMixIn):
    pass

class ProxyHandler(BaseHTTPServer.BaseHTTPRequestHandler):
    def do_GET(self):
        for pattern, url in FORWARD_LIST.items():
            match = re.search(pattern, self.path)
            if match:
                url = url % match.groups()
                break
        else:
            self.send_error(404)
            return
        dataobj = urllib.urlopen(url)
        data = dataobj.read()
        self.send_response(200)
        self.send_header("Content-Length", len(data))
        for key, value in dataobj.info().items():
            self.send_header(key, value)
        self.end_headers()
        self.wfile.write(data)

HTTPServer(("", 1234), ProxyHandler).serve_forever()
Community
  • 1
  • 1
Phillip
  • 13,448
  • 29
  • 41
  • Thanks! The script looks at least very promising. I think, I will at least try it with some real world example, if it fits my needs. When yes, I can also implement POST. The script looks a little like the tiniest proxy implementations I have seen, but I am/was not sure, if that will give a reliable solution. – Juergen Feb 27 '15 at 14:18
  • Did some small tests. The solution is not sufficient, at least because the result headers are not set. For example content-type is not passed threw. Also the response code is not set as in the original response. At least, some additional coding would be necessary, that it works in more cases. – Juergen Feb 27 '15 at 19:26
  • Bounty time expired, so I awarded it to the best answer until now. – Juergen Mar 06 '15 at 17:04
1

Your use case should be covered by: https://mitmproxy.org/doc/features/reverseproxy.html

Capt Planet
  • 113
  • 6
  • I don't think so. I already checked mitmproxy, but found no indication, that I can specify and select (e.g. by regex) different destination hosts. – Juergen Mar 04 '15 at 13:36
  • 1
    Someone asked for the exact same feature here: https://github.com/mitmproxy/mitmproxy/issues/486 – Capt Planet Mar 04 '15 at 16:08
  • Thanks for the link! The issue is still different than what I want to do (since the routing should be done on basis of host names, but what I want is on basis of the local paths), but could be, that the example helps me to get mitmproxy to do what I want, since it also includes a framework. I just hoped, that there is a already ready solution with no big programming and tweaking needed. – Juergen Mar 04 '15 at 16:23
  • I found the other answer a little more useful, so sorry for the bounty, but I upvoted your answer, since at least your comment contained a possibly helpful link. – Juergen Mar 06 '15 at 17:06
1

There is now a proxy that covers my needs (and more) -- very lightweight and very good:

Devd

Juergen
  • 12,378
  • 7
  • 39
  • 55