Understanding Long Polling Client Logic for Use With Azure Groovy Java

Question

Premise: We have groovy scripts that execute every minute. I want one of those scripts to open an HTTP client, and poll a service bus queue / topic for messages. I have my rest client code working an getting messages from the service bus queue. I can do a "Get" every 5 seconds, and wireshark shows that it's reusing the same TCP connection which is better than I expected, but its still not ideal.

Goal: I would like to make this http client do "long polling", for efficiency and to achieve actual real-time processing. It seems to be more complicated than I anticipated.

Problem: When I do a "Delete" call to read message from a service bus queue, it immediately returns "HTTP/1.1 204 No Content", and the connection closes. I set a timeout on the client, but I don't think that matters.

Here's the article that shows service bus says it's supports long polling, which I imagine is the hard part. Azure Service Bus Queues

I feel that I don't understand something fundamental about how to implement long polling in code. My understanding is that when there is no data in the queue, it's supposed to delay the response until data exists, or until my client eventually times out waiting (which lets me set my own disconnect/reconnect interval). I don't even care about blocking/nonblocking etc, because the script execution is already spreadout into a threadpool, and will be terminated forcibly and all that.

Any help is greatly appreciated.

Answer 1

The correct and simple answer is that adding the following to the end of an Azure REST API URL (with service bus) is the way to implements long-polling with that service: ?timeout=60 , where 60 tells azure to wait 60 seconds before responding with no-data. So, your application can check for data every 60 seconds, with an internal timeout of 60 seconds on each HTTP request. This will hold the TCP connection open for that timeframe, waiting for an HTTP response.

Answer 2

For understanding long polling, I recommend you can learn the entry Comet of Wiki https://en.wikipedia.org/wiki/Comet_(programming). And there is an answered thread (Long polling in java) explained the mechanism of the HttpURLConnection Class support long polling in Java.

As I know, a simple way in Java Client instead of HttpURLConnection is using the client library of CometD. You can refer to the section Client Library of its offical document https://docs.cometd.org/current/reference/#_java_client to learn how to implement the long polling client in Java. You can download the library at https://download.cometd.org/.

The sample code from the offical document:

// Create (and eventually set up) Jetty's HttpClient:
HttpClient httpClient = new HttpClient();
httpClient.start();

// Prepare the transport
Map<String, Object> options = new HashMap<String, Object>();
ClientTransport transport = new LongPollingTransport(options, httpClient);

// Create the BayeuxClient
ClientSession client = new BayeuxClient("http://localhost:8080/cometd", transport);

// Here set up the BayeuxClient, for example:
// client.getChannel(Channel.META_CONNECT).addListener(new ClientSessionChannel.MessageListener() { 
    public void onMessage(ClientSessionChannel channel, Message message) {
        if (message.isSuccessful()) {
            // Here handshake is successful
        }
    }
});
client.handshake();

Note: There are two REST API of Azure Service Bus for getting messaging entity(s) Get Entity https://msdn.microsoft.com/en-us/library/azure/hh780754.aspx and Entities Discovery https://msdn.microsoft.com/en-us/library/azure/hh780782.aspx. You need to delete the used messaging entity manually thru the Delete Entity REST API. Requesting all of these REST API first require an access_token thru the post request the Request a Token from ACS API for secure access.