Java: Using Apache HttpClient to send HttpGet to URI with underscore in host name

Question

I'm aware that underscores are technically not allowed in URI host names, but unfortunately we have customers who do that, so we need to support it. I've seen many other posts about this on StackOverflow where people are asking how to manually insert the host name into a URI object, and so I've already done that, but I'm still getting a SocketException when I try to send the request:

URI uri = new URI("https", "my_ftp.website.com", "/target.jpg", null, null);
MyUtils.setField(uri, "host", "my_ftp.website.com"); // manually set "host" through reflection
HttpGet httpGet = new HttpGet(uri);
HttpClients.createDefault().execute(httpGet);

As you can see in the code snippet above, I copied the instructions for manually setting the "host" field on the URI from this StackOverflow post, but it clearly isn't working. I even tried add the code from this StackOverflow post to the top of my program, but it didn't seem to have any effect either:

MyUtils.setField(URI.class, "L_DASH", 35184372088832L);
MyUtils.setField(URI.class, "H_DASH", 2147483648L);

No matter what I do, I always seem to get the same error:

java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(SocketInputStream.java:210)
    at java.net.SocketInputStream.read(SocketInputStream.java:141)
    at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
    at sun.security.ssl.InputRecord.read(InputRecord.java:503)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436)
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384)
    at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
    at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:374)
    at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:221)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:165)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:140)
    at com.amazon.emf.customerservice.activities.account.TempTest.runTest(TempTest.java:44)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:89)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:41)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:541)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:763)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:463)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:209)

Underscores are not legal in host names; and they will not work with Java URL or Apache HttpGet. Your first link talks about an entirely different opensource project that has a workaround to support host names with underscores. That is [this link](https://bitbucket.org/dzigoro/lid-commons/commits/22275d33e3d26b12f0a1d0563f058632d4baa938). Looks like his [project](https://bitbucket.org/dzigoro/lid-commons/src/default/). Best of luck! Have you tried [punycode](https://en.wikipedia.org/wiki/Punycode)? — Elliott Frisch, Feb 06 '20 at 20:13
In that link to the "LID commons" bitbucket project you mentioned, it looks like they are using apache: `import org.apache.http.client.methods.HttpGet`. What open source project do you think they're using? I haven't tried punycode -- looking at the wikipedia article you linked, I'm not clear on how that would help. Can you please clarify further? — Dasmowenator, Feb 06 '20 at 21:14
I checked, you can't punycode a `_` so I think you're stuck. How much control do you have over the dns infrastructure? Could you make `a.my_ftp.website.com` and have the `my_ftp.website.com` resolve to `a.my_ftp.website.com`? As far as I can tell, the issue is the `_` in the hostname (`_` in **domains** is fine). — Elliott Frisch, Feb 06 '20 at 21:28
We have no control over the DNS infrastructure. These URLs are provided to us by customers, so they're the ones who own these domains, not us. — Dasmowenator, Feb 06 '20 at 21:49
One possibility is to use java.net.URL and java.net.URLConnection. While java.net.URI checks for a valid URI syntax, java.net.URL is much more lax about it, probably because it is an ancient class that hails back to Java 1.0. — VGR, Feb 06 '20 at 22:39
I don't see how that would help. The HttpGet constructor only takes either a String or a URI, not a URL. Also, as I mentioned in my post, I was able to use reflection to manually set the host in the URI, and it still didn't fix it. — Dasmowenator, Feb 06 '20 at 22:43

Java: Using Apache HttpClient to send HttpGet to URI with underscore in host name

0 Answers0