16

I'm using elasticsearch and the RESTful API supports supports reading bodies in GET requests for search criteria.

I'm currently doing

response = urllib.request.urlopen(url, data).read().decode("utf-8")

If data is present, it issues a POST, otherwise a GET. How can I force a GET despite the fact that I'm including data (which should be in the request body as per a POST)

Nb: I'm aware I can use a source property in the Url but the queries we're running are complex and the query definition is quite verbose resulting in extremely long urls (long enough that they can interfere with some older browsers and proxies).

Basic
  • 26,321
  • 24
  • 115
  • 201
  • 4
    Going by Roy Fielding's [comment on the issue](http://tech.groups.yahoo.com/group/rest-discuss/message/9962) (via: http://stackoverflow.com/questions/978061/http-get-with-request-body), it seems like this is an abuse of the HTTP spec and the solution would be whapping the elasticsearch people for relying on this. Basically, the request body is allowed only insofar as a server shouldn't crash when it's present, not because it's meant to be processed in any way. (According to Wiki, Roy Fielding is one of the authors of HTTP, and the originator of REST, so I suppose he'd know.) – millimoose Mar 20 '13 at 16:55
  • @millimoose It's an intersting argument - semantically, it makes sense when considering a REST service - the GET relates to the logical action, not the communication mechanism – Basic Mar 20 '13 at 16:58
  • That said, I believe [`http.client`](http://docs.python.org/3.0/library/http.client.html) might be more flexible than `urllib` and allow you to construct "unusual" requests. – millimoose Mar 20 '13 at 17:00
  • I suppose this makes it more of a problem of the REST approach (i.e. hammering actions into the HTTP verbs when, in practice, they're not all "created equal") or of the HTTP spec (which could use a minor update to clarify the behaviour expected of implementations to support use cases that cropped up after 1.1 was finalized.) I'm reminded of the quote misattributed to Bill Gates. "256 query string chars should be enough for everybody." – millimoose Mar 20 '13 at 17:04
  • @millimoose You're not wrong - like the joy of trying to issue a DELETE using an html form. – Basic Mar 20 '13 at 17:10
  • Another way of looking at this is that this is, in fact, "not really REST". I belive that in "purist" REST, the results of a given "search" should not just be available over an idempotent GET of some form, but an honest-to-goodness addressable resource. Otherwise you're just exposing a search action and might as well make it a POST, which *happens to be* idempotent. To put it another way, idempotence is a necessary condition for a request to be a GET, not a sufficient one. – millimoose Mar 20 '13 at 17:13
  • Obviously, you could use the "SEARCH" method, and good luck having that work in any mainstream HTTP stack. Seems like with REST it's "damned if you do, damned if you don't". Doing it "right" is practically impossible, doing it "wrong" (i.e. doing RPC with idempotent read-only methods and a prettier URL scheme than SOAP gives you) makes you feel dirty and gets the semantic brigade on your back. – millimoose Mar 20 '13 at 17:14
  • @millimoose I think http.client might do the trick but I admit I'm now 50/50 as to which way to go. I'm tempted to implement both so I can change the method through config later. Some proxies barf on long query strings, others on GET bodies. – Basic Mar 20 '13 at 17:31

1 Answers1

25

I'm not aware of a nice way to do this using urllib. However, requests makes it trivial (and, in fact, trivial with any arbitrary verb and request content) by using the requests.request* function:

requests.request(method='get', url='localhost/test', data='some data')

Constructing a small test web server will show that the data is indeed sent in the body of the request, and that the method perceived by the server is indeed a GET.

*note that I linked to the requests.api.requests code because that's where the actual function definition lives. You should call it using requests.request(...)

Nisan.H
  • 6,032
  • 2
  • 26
  • 26