99

As stated in http://www.boutell.com/newfaq/misc/urllength.html, HTTP query string have limited length. It can be limited by the client (Firefox, IE, ...), the server (Apache, IIS, ...) or the network equipment (applicative firewall, ...).

Today I face this problem with a search form. We developed a search form with a lot of fields, and this form is sent to the server as a GET request, so I can bookmark the resulting page.

We have so many fields that our query string is 1100 bytes long, and we have a firewall that drops HTTP GET requests with more than 1024 bytes. Our system administrator recommends us to use POST instead so there will be no limitation.

Sure, POST will work, but I really feel a search as a GET and not a POST. So I think I will review our field names to ensure the query string is not too long, and if I can't I will be pragmatic and use POST.

But is there a flaw in the design of RESTful services? If we have limited length in GET request, how can I do to send large objects to a RESTful webservice? For example, if I have a program that makes calculations based on a file, and I want to provide a RESTful webservice like this: http://compute.com?content=<base64 file>. This won't work because the query string has not unlimited length.

I'm a little puzzled...

cbliard
  • 7,051
  • 5
  • 41
  • 47
  • 2
    What does restful mean in the terms of your context? Or paraphrasing: why GET is restful and POST isn't? Because GET can be constructed using simple string concatenation? Query length limitation is to avoid dynamic memory allocation in apps which are intended to work fast. – khachik Nov 17 '10 at 11:13
  • 7
    When I want to do a search, I do not want to create, delete or update something, I just want to retrieve data, so I should not use POST, DELETE or PUT, and I should use GET. This is how I understood REST but I may be mistaken about it – cbliard Nov 17 '10 at 11:18
  • GET is not suitable for searching, because the results of a search may change over time. Web infrastructure often allows caching of GET requests. If you use GET you risk getting old stale results for searches. POST is the way, as recommended below. – occulus May 31 '12 at 08:54
  • 17
    Everything changes overtime (http://en.wikipedia.org/wiki/Impermanence), such is the nature of the universe... But GET should be used for search because the "search action" does not change the results – Luxspes Oct 28 '12 at 21:20

7 Answers7

75

HTTP specification actually advises to use POST when sending data to a resource for computation.

Your search looks like a computation, not a resource itself. What you could do if you still want your search results to be a resource is create a token to identify that specific search result and redirect the user agent to that resource.

You could then delete search results tokens after some amount of time.

Example

POST /search
query=something&category=c1&category=c2&...

201 Created
Location: /search/01543164876

then

GET /search/01543164876

200 Ok
... your results here...

This way, browsers and proxies can still cache search results but you are submitting your query parameters using POST.

EDIT

For clarification, 01543164876 here represents a unique ID for the resource representing your search. Those 2 requests basically mean: create a new search object with these criteria, then retrieve the results associated with the created search object.

This ID can be a unique ID generated for each new request. This would mean that your server will leak "search" objects and you will have to clean them regularly with a caching strategy.

Or it can be a hash of all the search criteria actually representing the search asked by the user. This allows you to reuse IDs since recreating a search will return an existing ID that may (or may not) be already cached.

Vincent Robert
  • 35,564
  • 14
  • 82
  • 119
  • How does this address the OP requirement about bookmarking the query? – Rhubarb Nov 06 '14 at 09:44
  • 2
    @Rhubarb it does address it clearly by creating a resource for a given search. – maulik13 Nov 11 '14 at 10:21
  • 3
    This will slow down the results. Perform a post and then perform a GET. it will add atleast 300ms more to fetch the search. – jaxxbo Mar 22 '15 at 13:51
  • 1
    Link to source is dead – Emobe Mar 28 '17 at 20:08
  • I am confused, what does the server do in between the POST and the GET? does it cache the search data on the server and just wait for the GET request to come in? It would have to use some unique id to do that, so you could retrieve the cache. Your answer is nice, but surely not complete. And using a cache would make things less stateless. – Alexander Mills Jul 31 '17 at 17:54
  • `You could then delete search results tokens after some amount of time.` would you just delete the search results after the next applicable GET request came in? – Alexander Mills Jul 31 '17 at 17:55
  • @AlexanderMills No, your GET request should be safe, so any client must feel free to query it multiple times without any side effect. The resource disappearing after the first request would be a side effect of this request and would not make this request safe. Deleting after some amount of time (1 hour, 1 day, 1 week) depending on your activity is OK as long as you can send a 410 Gone response after the deletion. – Vincent Robert Aug 07 '17 at 13:16
  • Thanks Vincent, I only follow like 50% of what you mean - do you have a link describing how to do this right, or is there some term used to describe it so I can search for it? – Alexander Mills Aug 07 '17 at 21:04
  • You can start with the Wikipedia article on REST that really clarifies the meaning of each HTTP methods and how to use them. It will also help you learn the vocabulary (safe = no side-effects, idempotent = repeatable, etc.) : https://en.wikipedia.org/wiki/Representational_state_transfer – Vincent Robert Aug 08 '17 at 21:18
  • This is by far the best answer. It didn't occur to me that the search is being created and not just returned. Thank you for making this make sense for me! – T0t3sMcG0t3s Jul 01 '19 at 17:50
58

Based on your description, IMHO you should use a POST. POST is for putting data on the server and, in some cases, obtain an answer. In your case, you do a search (send a query to the server) and get the result of that search (retrieve the query result).

The definition of GET says that it must be used to retrieve an already existing resource. By definition, POST is to create a new resource. This is exactly what you are doing: creating a resource on the server and retrieving it! Even if you don't store the search result, you created an object on the server and retrieved it. As PeterMmm previsouly said, you could do this with a POST (create and store the query result) and then use a GET to retrive the query, but it's more pratical do only a POST and retrieve the result.

Hope this helps! :)

jmpcm
  • 1,814
  • 2
  • 19
  • 27
  • 2
    You are right, I can see it as a POST because the search is a volatile newly computed resource. But I still have problems seeing the boundary between POST and GET. If I want to search all science-fiction books in a library, I will get a collection of existing resources, so I am tempted to use a GET, but you propose to see it as a POST because the search is a new resource itself. So the query string in GET should be used only to change data representation, but not to filter data. Am I right? – cbliard Nov 17 '10 at 11:36
  • Yes, if you look from the perspective of the search, she is a resource itself that you must create (do a POST) in order to retrieve some data (a GET). You can see a search as a new object instantiated from a collection of already existing objects! Using the library example, you have the books (resources) on the shelves, but you only want a part of them and this part can be seen as a new resource (think of a new shelf only with the books you search). – jmpcm Nov 17 '10 at 11:54
  • @jmpcm To be clear, you're saying that any GET parameters which are simply specifying which exact resource to get (i.e. `/?file=123.txt`) are the only proper ways to use GET, and that POST should be used for anything searchy (i.e. `/?file=*.txt`), or am I misunderstanding? – Patrick Sep 13 '12 at 16:16
  • @Patrick: GET can be used to retrieve more than one existing resource, i.e. your GET request could have a query like `/?file1=f1.txt&file2=f2.txt` in order to retrieve 2 files. If you are searching for something more "unspecified", than you should use a POST because an object with the result will be created on the server and is sent to the client. Conclusion: you can put things the way you wrote above, but understand that it can be misleading, specially in the second case where you can have on the server an existing result resource with the result; in this case GET could be an option. – jmpcm Sep 16 '12 at 22:21
  • 10
    POST should not be use for searches, GET is what should be used, this way you can share, and cache, the resulting URL and the results, and you take better advantage of the RESTful architecture of the internet – Luxspes Oct 26 '12 at 04:50
  • 1
    The question to ask is: does this change server state? If the server needs to remember the request to properly fill out future requests, then the server state has changed and either POST, PUT, or DELETE should be used. If the server does not need to remember the request in order to properly fill out future requests then GET should be used. Server side caching and hit counters don't count as changing server state. – jmh Sep 27 '13 at 19:47
  • 47
    This answer sounds more like trying to play with words. By that logic, we can make everything as POST requests and still be RESTful. – supertonsky Jul 30 '14 at 03:10
  • How does this address the OP requirement about bookmarking the query? – Rhubarb Nov 05 '14 at 18:12
  • 10
    This answer seems like some lawyer talk to me :-) The question is really "I do have a GET case, okay. But my query string exceeds the allowed length. How do I handle". Downvoting this answer. – G. Stoynev Feb 26 '15 at 03:26
  • 1
    @G.Stoynev: I know it is a bit philosophical, but isn't what life is all about? ;) This is the gray area of REST web services, but still there is recommendations to them. Check Vicent Robert's answer bellow; it is very pragmatic and straight to the point. – jmpcm Mar 04 '15 at 07:58
  • 3
    If that logic followed, then every request is creating a resource and retrieving it. – Alex Oct 12 '15 at 17:40
5

REST is a manner to do things, not a protocol. Even if you dislike to POST when it is really a GET, it will work.

If you will/must stay with the "standard" definition of GET, POST, etc. than maybe consider to POST a query, that query will be stored on the server with a query id and request the query later with GET by id.

PeterMmm
  • 24,152
  • 13
  • 73
  • 111
5

Regarding your example:http://compute.com?content={base64file}, I would use POST because you are uploading "something" to be computed. For me this "something" feels more like a resource as a simple parameter.

In contrast to this in usual search I would start to stick with GET and parameters. You make it so much easier for api-clients to test and play around with your api. Make the read-only access (which in most cases is the majority of traffic) as simple as possible!

But the dilemma of large query strings is a valid limitation of GET. Here I would go pragmatic, as long as you don't hit this limit go with GET and url-params. This will work in 98% of search-cases. Only act if you hit this limit and then also introduce POST with payload (with mime-type Content-Type: application/x-www-form-urlencoded).

Have you got more real-world examples?

manuel aldana
  • 15,650
  • 9
  • 43
  • 50
  • The computation was a real world example. About the search, we want to be able to search for transactions on multiple sell points, so we open a new browser window to select the sell points. And when we validate we modify a hidden parameter in the search form to set the selected sell points. If there are really many of them, then the resulting search request will have a very long query string. – cbliard Nov 18 '10 at 12:59
3

The confusion around GET is a browser limitation. If you are creating a RESTful interface for an A2A or P2P application then there is no limitation to the length of your GET.

Now, if you happen to want to use a browser to view your RESTful interface (aka during development/debugging) then you will run into this limit, but there are tools out there to get around this.

ken kranz
  • 63
  • 1
  • 1
  • 9
    "GET is a browser limitation" -- it is also a server limitation. You will find all web servers enforce limit, and if you have CDNs they may also enforce a limit. Using other tools to perform the request will not circumvent the server limits. – Courtney Miles Sep 20 '18 at 01:55
1

This is an easy one. Use POST. HTTP doesn't impose a limit on the URL length for GET but servers do. Be pragmatic and work around that with a POST.

You could also use a GET body (that is allowed) but that's a double-whammy in that it is not correct usage and probably going to have server problems.

Rick O'Shea
  • 1,410
  • 19
  • 15
0

I think if u develop the biz system, encounter this issue, u must think whether the api design reasonable, if u GET api param design a biz_ids, and it too long.

u should think about with UI or Usecase, whether use other_biz_id to find biz_ids and build target response instead of biz_ids directly or not.

if u old api be depended on, u can add a new api for this usecase, if u module design well u add this api may fast.

I think should use protocols in a standard way as developer. hope help u.

27_zZ
  • 1
  • 1