3

I looked and found the answer to some people having issue with GET request URL exceeding the maximum length in Jetty is to set the headerBufferSize in jetty.xml to be a bigger number as in this Solr troubleshooting manual and this.

However, I have a hard time to understand what the header buffer size has to do with the request URL's length? If setting headerBufferSize increase request's URL length limit, what does a value of 6 KB to headerBufferSize correspond to the maximum length of the request's URL? The reason I ask because the maximum length of URL imposed by most browsers is around 2000 characters as in What is the maximum length of a URL in different browsers? and headerBufferSize's unit is in Bytes.

Community
  • 1
  • 1
P. Hoang
  • 451
  • 3
  • 6
  • 12

1 Answers1

12

In a typical POST request you will see the following ...

POST /to/my/path HTTP/1.1
Content-Type: application/x-www-form-urlencoded;charset=utf-8
Host: https://my.machine.com
Content-Length: 10

Action=Add

Breaking this down:

  • The POST through Content-Length lines are the Request Headers.
    • The POST line is known, in HTTP terminology, as the Request-Line, it contains the method (POST) + abs_path (/to/my/path) + http version (HTTP/1.1)
    • Content-Type - lets us know how the body content is formatted/encoded.
    • Host - lets the server know what host was being accessed (used mainly by virtual host setups)
    • Content-Length - lets us know that there is 10 bytes of body content
  • The Action=Add is the POST body content.

At its heart there are 2 parts of a request or response, the Headers and the Body Content.

When you set the headerBufferSize you are setting the ultimate upper limit for the header content (not body content).

There are a number of abuses / vulnerabilities present when you have unlimited header sizes, ranging from abusive memory consumption, to intentional hashmap collisions resulting in excessive CPU use. Limiting the header buffer sizes limits the scope of these kinds of issues. (these vulnerabilities are not unique to Jetty, but exist for all web servers)

If you are hitting these limits, you should consider evaluating how you are using solr (such as incorrectly using GET when you should be using POST), as increasing the headerBufferSize will also open you up to the various known web vulnerabilities.

Update: Oct 24, 2013

See other answer related to What is the maximum length of a URL

Community
  • 1
  • 1
Joakim Erdfelt
  • 46,896
  • 7
  • 86
  • 136
  • I'm not thinking about increasing the `headerBufferSize` but more curious to know what is the maximum length of the GET Request URL. Based on your answer, I assume the only way to know the maximum allowed size of the URL have to be calculated by taking the maximum header buffer size (allowed by Jetty) and subtract the header's size (without the request URL). Is it correct? – P. Hoang Oct 24 '13 at 14:10
  • Added update to answer pointing at other, more detailed, answer about URL length. (that question is browser specific, but the answer isn't) – Joakim Erdfelt Oct 24 '13 at 15:34
  • 1
    Thanks for the info. To be clear, I was asked at work to find out the maximum length of the request URL that Jetty can accept. Since my Jetty server accepts requests either made from the browser and made programatically, I have to document the maximum length for URL the GET method that the user can safely use to send request to my server. If I give out a big number, they may run into issue if the size of the request URL + headers they add to the request exceed the allowed `headerBufferSize`. Now, think about it, I may just give out the value of `headerBufferSize` and let the user decide. – P. Hoang Oct 24 '13 at 17:23
  • 1
    Jetty can support URL sizes up to whatever memory you have on your server. You can easily configure the headerBufferSize to something like 10MB or 100MB or even 1GB. In a more practical sense, if you have both browsers and library based clients, settle on 2000 characters as that's what the rest of the world has settled on. – Joakim Erdfelt Oct 24 '13 at 18:17
  • Great to know, I will do as you suggested then. – P. Hoang Oct 24 '13 at 18:45