Vertx HttpClient getNow not working

Question

I have problem with vertx HttpClient. Here's code which shows that tests GET using vertx and plain java.

    Vertx vertx = Vertx.vertx();
    HttpClientOptions options = new HttpClientOptions()
            .setTrustAll(true)
            .setSsl(false)
            .setDefaultPort(80)
            .setProtocolVersion(HttpVersion.HTTP_1_1)
            .setLogActivity(true);
    HttpClient client = vertx.createHttpClient(options);

    client.getNow("google.com", "/", response -> {
        System.out.println("Received response with status code " + response.statusCode());
    });
    System.out.println(getHTML("http://google.com"));

Where getHTML() is from here: How do I do a HTTP GET in Java?

This is my output:

<!doctype html><html... etc <- correct output from plain java
Feb 08, 2017 11:31:21 AM io.vertx.core.http.impl.HttpClientRequestImpl
SEVERE: java.net.UnknownHostException: failed to resolve 'google.com'.     Exceeded max queries per resolve 3

But vertx can't connect. What's wrong here? I'm not using any proxy.

Might related to https://github.com/eclipse/vert.x/issues/1753. Can you try with Vert.x 3.4.0.Beta1 ? It should be fixed in that version. Also, you can fallback to the JVM resolver with `-Dvertx.disableDnsResolver=true`. — tsegismont, Feb 08 '17 at 11:23
I tried both 3.3.3 and 3.4.0.Beta1 and it's not working. I'm gonna check this jvm parameter. — Wojciech Kumoń, Feb 08 '17 at 11:57
-Dvertx.disableDnsResolver=true works :) Please answer this question (I'll accept that). — Wojciech Kumoń, Feb 08 '17 at 12:11
It's weird your example works fine for me. What is your DNS env? — tsegismont, Feb 08 '17 at 15:21
Can you capture the DNS frames with Wireshark and report an issue on GitHub? — tsegismont, Feb 08 '17 at 16:04
remove the scheme in System.out.println(getHTML("http://google.com")); to System.out.println(getHTML("google.com")); — Lucas Milotich, Sep 27 '18 at 10:59

score 3 · Answer 1 · answered Nov 26 '19 at 15:50

For reference: a solution, as described in this question and in tsegismont's comment here, is to set the flag vertx.disableDnsResolver to true:

-Dvertx.disableDnsResolver=true

in order to fall back to the JVM DNS resolver as explained here:

sometimes it can be desirable to use the JVM built-in resolver, the JVM system property -Dvertx.disableDnsResolver=true activates this behavior

I observed this DNS resolution issue with a redis client in a kubernetes environment.

score 3 · Answer 2 · answered May 09 '20 at 12:03

I had this issue, what caused it for me was stale DNS servers being picked up by the Java runtime, i.e. servers registered for a network the machine was no longer connected to. The issue is first in the Sun JNDI implementation, it also exists in Netty which uses JNDI to bootstrap its list of name servers on most platforms, then finally shows up in VertX.

I think a good place to fix this would be in the Netty layer where the set of default DNS servers is bootstrapped. I have raised a ticket with the Netty project so we'll see if they agree with me! Here is the Netty ticket

In the mean time a fairly basic workaround is to filter the default DNS servers detected by Netty, based on whether they are reachable or not. Here is a code Sample in Kotlin to apply before constructing the main VertX instance.

// The default set of name servers provided by JNDI can contain stale entries
// This default set is picked up by Netty and in turn by VertX
// To work around this, we filter for only reachable name servers on startup

val nameServers = DefaultDnsServerAddressStreamProvider.defaultAddressList()

val reachableNameServers = nameServers.stream()
        .filter {ns -> ns.address.isReachable(NS_REACHABLE_TIMEOUT)}
        .map {ns -> ns.address.hostAddress}
        .collect(Collectors.toList())

if (reachableNameServers.size == 0)
    throw StartupException("There are no reachable name servers available")

val opts = VertxOptions()
opts.addressResolverOptions.servers = reachableNameServers

// The primary Vertx instance
val vertx = Vertx.vertx(opts)

A little more detail in case it is helpful. I have a company machine, which at some point was connected to the company network by a physical cable. Details of the company's internal name servers were set up by DHCP on the physical interface. Using the wireless interface at home, DNS for the wireless interface gets set to my home DNS while the config for the physical interface is not updated. This is fine since that device is not active, ipconfig /all does not show the internal company DNS servers. However, looking in the registry they are still there:

Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces

They get picked up by the JNDI mechanism, which feeds Netty and in turn VertX. Since they are not reachable from my home location, DNS resolution fails. I can imagine this home/office situation is not unique to me! I don't know whether something similar could occur with multiple virtual interfaces on containers or VMs, it could be worth looking at if you are having problems.

score 0 · Answer 3 · answered Apr 13 '17 at 23:30

Here is the sample code which works for me.

public class TemplVerticle extends HttpVerticle {

public static void main(String[] args) {
    Vertx vertx = Vertx.vertx();

    // Create the web client and enable SSL/TLS with a trust store
    WebClient client = WebClient.create(vertx,
            new WebClientOptions()
                    .setSsl(true)
                    .setTrustAll(true)
                    .setDefaultPort(443)
                    .setKeepAlive(true)
                    .setDefaultHost("www.w3schools.com")

    );


    client.get("www.w3schools.com")

            .as(BodyCodec.string())
            .send(ar -> {
                if (ar.succeeded()) {
                    HttpResponse<String> response = ar.result();
                    System.out.println("Got HTTP response body");
                    System.out.println(response.body().toString());

                } else {
                    ar.cause().printStackTrace();
                }
            });
}

}

score 0 · Answer 4 · answered Oct 03 '18 at 00:04

Try using web client instead of httpclient, here you have an example (with rx):

private val client: WebClient = WebClient.create(vertx, WebClientOptions()
        .setSsl(true)
        .setTrustAll(true)
        .setDefaultPort(443)
        .setKeepAlive(true)
)

open fun <T> get(uri: String, marshaller: Class<T>): Single<T> {
    return client.getAbs(host + uri).rxSend()
            .map { extractJson(it, uri, marshaller) }
}

Another option is to use getAbs.

Vertx HttpClient getNow not working

4 Answers4

Linked