Java 9 HttpClient exception when using certain characters in URL query parameters

Question

Here is my sample code. The query is encoded to UTF-8:

HttpRequest request = HttpRequest.newBuilder()
    .header("content-type", "application/json;charset=UTF-8")
    .uri(URI.create("http://localhost:8080/test?param1=test%C5%84"))
    .GET()
    .build();

HttpClient.newBuilder()
    .version(HttpClient.Version.HTTP_2)
    .build()
    .send(request, HttpResponse.BodyHandler.asString(Charset.forName("UTF-8")));

After I run this example I get the following exception:

java.lang.IllegalArgumentException: char=324
at jdk.incubator.httpclient/jdk.incubator.http.internal.hpack.Huffman.codeOf(Huffman.java:559)
at jdk.incubator.httpclient/jdk.incubator.http.internal.hpack.Huffman.lengthOf(Huffman.java:524)
at jdk.incubator.httpclient/jdk.incubator.http.internal.hpack.StringWriter.configure(StringWriter.java:79)
at jdk.incubator.httpclient/jdk.incubator.http.internal.hpack.StringWriter.configure(StringWriter.java:62)
at jdk.incubator.httpclient/jdk.incubator.http.internal.hpack.IndexNameValueWriter.value(IndexNameValueWriter.java:64)
at jdk.incubator.httpclient/jdk.incubator.http.internal.hpack.Encoder.literal(Encoder.java:422)
at jdk.incubator.httpclient/jdk.incubator.http.internal.hpack.Encoder.header(Encoder.java:245)
at jdk.incubator.httpclient/jdk.incubator.http.internal.hpack.Encoder.header(Encoder.java:198)
at jdk.incubator.httpclient/jdk.incubator.http.Http2Connection.encodeHeadersImpl(Http2Connection.java:927)
at jdk.incubator.httpclient/jdk.incubator.http.Http2Connection.encodeHeaders(Http2Connection.java:878)
at jdk.incubator.httpclient/jdk.incubator.http.Http2Connection.encodeHeaders(Http2Connection.java:951)
at jdk.incubator.httpclient/jdk.incubator.http.Http2Connection.sendFrame(Http2Connection.java:984)
at jdk.incubator.httpclient/jdk.incubator.http.Stream.sendHeadersAsync(Stream.java:547)
at jdk.incubator.httpclient/jdk.incubator.http.Exchange.lambda$responseAsyncImpl0$8(Exchange.java:322)
at java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1072)
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SSLFlowDelegate.setALPN(SSLFlowDelegate.java:164)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SSLFlowDelegate.access$200(SSLFlowDelegate.java:81)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SSLFlowDelegate$Reader.processData(SSLFlowDelegate.java:340)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SSLFlowDelegate$Reader$ReaderDownstreamPusher.run(SSLFlowDelegate.java:215)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler$SynchronizedRestartableTask.run(SequentialScheduler.java:175)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:147)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler$TryEndDeferredCompleter.complete(SequentialScheduler.java:315)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:149)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:198)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:271)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:224)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SSLFlowDelegate$Reader.incoming(SSLFlowDelegate.java:242)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SubscriberWrapper.incomingCaller(SubscriberWrapper.java:388)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SubscriberWrapper.onNext(SubscriberWrapper.java:343)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SubscriberWrapper.onNext(SubscriberWrapper.java:58)
at jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$InternalReadSubscription.read(SocketTube.java:739)
at jdk.incubator.httpclient/jdk.incubator.http.SocketTube$SocketFlowTask.run(SocketTube.java:171)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:198)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:271)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:224)
at jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$InternalReadSubscription.signalReadable(SocketTube.java:675)
at jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$ReadEvent.signalEvent(SocketTube.java:829)
at jdk.incubator.httpclient/jdk.incubator.http.SocketTube$SocketFlowEvent.handle(SocketTube.java:243)
at jdk.incubator.httpclient/jdk.incubator.http.HttpClientImpl$SelectorManager.handleEvent(HttpClientImpl.java:769)
at jdk.incubator.httpclient/jdk.incubator.http.HttpClientImpl$SelectorManager.run(HttpClientImpl.java:731)

char=324 means decoded ń from query

When I read stack trace I found jdk.incubator.http.Stream<T> this method in the class:

private void setPseudoHeaderFields() {
    HttpHeadersImpl hdrs = requestPseudoHeaders;
    String method = request.method();
    hdrs.setHeader(":method", method);
    URI uri = request.uri();
    hdrs.setHeader(":scheme", uri.getScheme());
    // TODO: userinfo deprecated. Needs to be removed
    hdrs.setHeader(":authority", uri.getAuthority());
    // TODO: ensure header names beginning with : not in user headers
    String query = uri.getQuery();
    String path = uri.getPath();
    if (path == null || path.isEmpty()) {
        if (method.equalsIgnoreCase("OPTIONS")) {
            path = "*";
        } else {
            path = "/";
        }
    }
    if (query != null) {
        path += "?" + query;
    }
    hdrs.setHeader(":path", path);
}

In this method uri.getQuery() is used which gives us the decoded query and causes the above exception.

When I used uri.getRawQuery() in debug mode (which gave us the encoded query) everything went fine.

My question is: is that a bug or intentional usage? If it is not a bug, how do I avoid the exception?

`test%C5%84` maybe (extra %)? As in `"...?param=" + URLEncoder.encode("ń", "UTF-8")` — Joop Eggen, Jun 14 '18 at 11:06
Sorry, it is my mistake. There should be `test%C5%84`, but in method `setPseudoHeaderFields()` I get `...testń`, and future exception. — lassa, Jun 14 '18 at 11:10
It seems that the Huffman compression uses a simplistic char to byte conversion à la ISO-8859-1. So to my very limited experience it seems a _bug_. You might revert to Base64: `"...?param1=" + Base64.getEncoder().encode("ń")` but then you need to decode the parameter yourself too. — Joop Eggen, Jun 14 '18 at 11:30

Michael · Accepted Answer · 2018-06-14T12:34:11.107

13

It is a bug:

java.lang.IllegalArgumentException with jdk.incubator.httpclient when using some UTF-8 characters

Some UTF-8 characters like "š" can't be used with HttpClient. However others like "å" can be.

See: https://bugs.openjdk.java.net/browse/JDK-8201238

Java 9/10 HttpClient is incubated. It's not production-ready. It states so very clearly in the package name. As such, it's quite possible that it may contain bugs.

It will be released properly in Java 11 (currently scheduled for release 25th September 2018) as part of JEP 321.

edited Jun 14 '18 at 12:34

answered Jun 14 '18 at 11:57

Michael

41,989
11
82
128

8

Note: the HTTP Client API will be standardized in Java 11 and will be moved to a new package (and module) named `java.net.http`. ([JEP 321](http://openjdk.java.net/jeps/321)) – Slaw Jun 14 '18 at 12:11
1

@Slaw I am so happy for this, finally a standard http client in the jdk itself! – Eugene Jun 14 '18 at 13:34
1

You can plan to [migrate to Java-11 for using the **standardized `http` module.**](https://stackoverflow.com/questions/50122696/the-jdk-incubator-httpclient-module-not-found-in-java11). I can confirm that the [code in question](https://github.com/namannigam/jdk11-updates/blob/master/src/main/java/com/stackoverflow/nullpointer/http/HttpBug8201238.java) works appropriately with JDK/11 (jdk-11-ea+17). – Naman Jun 14 '18 at 18:31

Java 9 HttpClient exception when using certain characters in URL query parameters

1 Answers1