6

Using the AsyncHttpClient with Netty provider will prevent the main program to terminate when we execute an asynchronous request. For instance, the following program terminates after the println, or not, depending on whether the provider is JDKAsyncHttpProvider or NettyAsyncHttpProvider:

public class Program {
    public static CompletableFuture<Response> getDataAsync(String uri) {
        final AsyncHttpClient asyncHttpClient = new AsyncHttpClient();
        final CompletableFuture<Response> promise = new CompletableFuture<>();
        asyncHttpClient
            .prepareGet(uri)
            .execute(new AsyncCompletionHandler<Response>(){
                @Override
                public Response onCompleted(Response resp) throws Exception {
                    promise.complete(resp);
                    asyncHttpClient.close(); // ??? Is this correct ????
                    return resp;
                }
            });
        return promise;
    }

    public static void main(String [] args) throws Exception {
        final String uri = "…";
        System.out.println(getDataAsync(uri).get());
    }
}

About the AsynHttpClient the documentation states:

AHC is an abstraction layer that can work on top of the bare JDK, Netty and Grizzly. Note that the JDK implementation is very limited and you should REALLY use the other real providers.

To use AsyncHttpClient with Netty we just need to include the corresponding library in the java class path. So, we may run the previous Program with one of the following class path configurations to use Netty, or not:

  • -cp .;async-http-client-1.9.24.jar;netty-3.10.3.Final.jar;slf4j-api-1.7.12.jar will use NettyAsyncHttpProvider
  • -cp .;async-http-client-1.9.24.jar;slf4j-api-1.7.12.jar will use JDKAsyncHttpProvider

What else should we do to use Netty provider correctly? For instance, I am closing the AsyncHttpClient in AsyncCompletionHandler. Is that correct?

Is there any configuration to change the observed behavior?

Miguel Gamboa
  • 8,855
  • 7
  • 47
  • 94
  • Isn't `AsyncHttpClient` thread-safe? Don't create a new one on each method invocation. – Sotirios Delimanolis May 29 '15 at 16:35
  • if you try closing the client much later, e.g. after `thenAccept`, would your application terminate? – ZhongYu May 29 '15 at 16:36
  • No. Closing, or not, the asyncHttpClient has no influence in the resulting behavior. – Miguel Gamboa Jun 01 '15 at 10:14
  • When I try your code, i get this error, maybe it's a deadlock issue as it says. But you did say you tried it without the close? WARN [New I/O worker #1] (NettyAsyncHttpProvider.java:79) - Unexpected error on close java.lang.IllegalStateException: Must not be called from a I/O-Thread to prevent deadlocks! at org.jboss.netty.channel.socket.nio.AbstractNioSelector.shutdown(AbstractNioSelector.java:415) – Alper Akture Jun 02 '15 at 20:06
  • @Alper Akture, I tested my code with Jdk8_u45 on both Windows 7 and Ubuntu 10.04.4 LTS, kernel version 2.6.32-66-server and I got the same behavior. None of the environments raise any exception and the programa terminates, or not, just depending on whether I include the `netty-3.10.3.Final.jar` in the classpath, or not. Thanks – Miguel Gamboa Jun 04 '15 at 16:06
  • 1
    I get the same behavior, but if I move the close() to after the System.out.println, it does exit when I include netty jar. But you tried moving the close, and it still doesn't exit? – Alper Akture Jun 04 '15 at 18:57
  • Looks like the issue is due to this [post](https://groups.google.com/forum/#!topic/netty/b0AK929RgPs). When I do a thread dump of your program, I see the same "Hashed wheel timer" non daemon thread. – Alper Akture Jun 04 '15 at 19:34
  • Yes @Alper Akture, you are right the Program finishes properly when I move the close() operation to the main() method. So, I should not close() the client in the callback. I will update the title of my post to reflect my mistake. thanks – Miguel Gamboa Jun 05 '15 at 09:25
  • @MiguelGamboa Can you explain how you are closing it in main method? – AVINASH SHRIMALI May 30 '18 at 07:01
  • @AVINASHSHRIMALI The main method is blocking on the result of `getDataAsync(uri)` through the call to `get()` that will wait for the promise completion, i.e. `promise.complete(resp)`. There is a synchronization point between the call to the `get()` in the `main` and the call to the `promise.complete(resp)`. The later notifies the former to proceed. After that it closes the `asyncHttpClient`. In truth maybe I should switch the order between the statements `promise.complete(resp)` and `asyncHttpClient.close()`. – Miguel Gamboa May 30 '18 at 14:49

1 Answers1

5

Using the netty provider and stepping through it in the debugger, I see that calling asyncHttpClient.close() inside the callback causes NioSocketChannelFactory.releaseExternalResources() to fail when it tries to do its shutdown. An exception is thrown, and this is likely causing the non daemon thread to remain, and keeping the vm from exiting. The exception thrown is logged at WARN, so you're probably not seeing it (if you add slf4j-log4j12-1.7.7.jar to you classpath you should see it). So you can't call close() in the call back (and you don't need to close it after every request execution anyhow).

Here's the trace:

 WARN [New I/O worker #1] (NettyAsyncHttpProvider.java:79) - Unexpected error on close
java.lang.IllegalStateException: Must not be called from a I/O-Thread to prevent deadlocks!
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.shutdown(AbstractNioSelector.java:415)
at org.jboss.netty.channel.socket.nio.NioWorker.shutdown(NioWorker.java:36)
at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.shutdown(AbstractNioWorkerPool.java:142)
at org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory.releaseExternalResources(NioClientSocketChannelFactory.java:225)
at com.ning.http.client.providers.netty.channel.ChannelManager.close(ChannelManager.java:355)
at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.close(NettyAsyncHttpProvider.java:70)
at com.ning.http.client.AsyncHttpClient.close(AsyncHttpClient.java:336)
at com.cie.program.Program$1.onCompleted(Program.java:23)
Alper Akture
  • 2,445
  • 1
  • 30
  • 44