Tomcat 9: header line does not conform to RFC 7230

Question

I have an app that was working on Tomcat 8.5.38. Now I decided upgrade to Tomcat 9.0.27 and there appears problem with GET request and RFC 7230, Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing.

The request:

/api/vehicle/power_off?vehicleId=1428714&dtStart=2019-10-21 08:00:00&dtEnd=2019-10-21 08:30:00

It was working perfectly from both browser (any - IE, Opera, Chrome, FF) and another client (1C ERP system).

After version upgrade from browser it still works perfectly but from 1C don't. Tomcat shows error:

28-Oct-2019 17:29:26.201 INFO [http-nio-8080-exec-3] org.apache.coyote.http11.Http11Processor.service Error parsing HTTP request header
 Note: further occurrences of HTTP request parsing errors will be logged at DEBUG level.
    java.lang.IllegalArgumentException: The HTTP header line [get /api/vehicle/power_off?deviceId=1428714&dtStart=2019-10-21%2008:00:00&dtEnd=2019-10-21%2008:30:00 HTTP/1.1: ] does not conform to RFC 7230 and has been ignored.
        at org.apache.coyote.http11.Http11InputBuffer.skipLine(Http11InputBuffer.java:962)
        at org.apache.coyote.http11.Http11InputBuffer.parseHeader(Http11InputBuffer.java:825)
        at org.apache.coyote.http11.Http11InputBuffer.parseHeaders(Http11InputBuffer.java:564)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:309)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:860)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1587)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

The same error is on my dev machine (MacOS + Tomcat 9.0.24) and production server (Ubuntu 16.04 + Tomcat 9.0.27).

The reason is in colon in datetime parameters. When I remove colons from query string (leave just "2019-10-21 080000") the request works as expected (with error "datetime cannot be parsed..."). Also when I manually change colons to "%3A" the request works and returns normal result.

Then I add relaxedQueryChars parameter to Tomcat Connector with colon (though colon is allowed symbol):

relaxedQueryChars=':[]|{}^\`"<>'

and it still fails.

What's the difference between 8 and 9 Tomcat versions that my request works in 8 but in 9 do not? Is there anything I can do in Tomcat to make this request works? Changing requests on client sides is very hard task...

Answer 1

What's the difference between 8 and 9 Tomcat versions that my request works in 8 but in 9 do not?

I think the difference is that Tomcat 9.x has tightened up on what should be permitted to be unencoded within a URL, so from a technical perspective there is no problem with Tomcat 9.x; the issue lies with earlier Tomcat releases, and browsers not strictly following specifications.

That said, I couldn't identify any specific fix that has triggered this issue for you, nor could I see anything in the Release Notes.

I add relaxedQueryChars parameter to Tomcat Connector with colon (though colon is allowed symbol)...and it still fails.

From the Tomcat 9.0 documentation for relaxedQueryChars:

The HTTP/1.1 specification requires that certain characters are %nn encoded when used in URI query strings. Unfortunately, many user agents including all the major browsers are not compliant with this specification and use these characters in unencoded form. To prevent Tomcat rejecting such requests, this attribute may be used to specify the additional characters to allow. If not specified, no additional characters will be allowed. The value may be any combination of the following characters: " < > [ \ ] ^ ` { | } . Any other characters present in the value will be ignored.

Note the last two sentences. The colon character is not mentioned, so it "will be ignored".

Is there anything I can do in Tomcat to make this request work?

I don't think so, but the real problem is that you are not encoding colons within your parameters, and you have already mentioned that this resolves the issue. See this SO answer, and in particular the final sentence:

There are reserved characters, that have a reserved meanings, those are delimiters — :/?#[]@ — and subdelimiters — !$&'()*+,;=

There is also a set of characters called unreserved characters — alphanumerics and -._~ — which are not to be encoded.

That means, that anything that doesn't belong to unreserved characters set is supposed to be %-encoded, when they do not have special meaning (e.g. when passed as a part of GET parameter).

The colon is a reserved character with a special meaning, and therefore it must be encoded within your parameters.

Notes:

• Also see Bug 62273 - Add support for alternate URL specification. Although it doesn't specifically address your issue with the colon character, there is an interesting discussion on how browsers have not been adhering to RFC 3986 (Uniform Resource Identifier (URI): Generic Syntax).

• The error message you are getting from Tomcat is vague, and could probably be improved. Perhaps raise a bug report with them?

Answer 2

In my case, I had increased the request header size by manually adding new cookies onto the Curl request. In doing so, ENTER key was used instead of space. Backspacing & using a space instead resolved the issue for me.