How to create a Java non-blocking InputStream from a HttpsURLConnection?

Question

Basically, I have a URL that streams xml updates from a chat room when new messages are posted. I'd like to turn that URL into an InputStream and continue reading from it as long as the connection is maintained and as long as I haven't sent a Thread.interrupt(). The problem I'm experiencing is that BufferedReader.ready() doesn't seem to become true when there is content to be read from the stream.

I'm using the following code:

BufferedReader buf = new BufferedReader(new InputStreamReader(ins));


String str = "";
while(Thread.interrupted() != true)
{
    connected = true;
    debug("Listening...");

    if(buf.ready())
    {
        debug("Something to be read.");
        if ((str = buf.readLine()) != null) {
            // str is one line of text; readLine() strips the newline character(s)
            urlContents += String.format("%s%n", str);
            urlContents = filter(urlContents);
        }
    }

    // Give the system a chance to buffer or interrupt.
    try{Thread.sleep(1000);} catch(Exception ee) {debug("Caught thread exception.");}
}

When I run the code, and post something to the chat room, buf.ready() never becomes true, resulting in the lines never being read. However, if I skip the "buf.ready()" part and just read lines directly, it blocks further action until lines are read.

How do I either a) get buf.ready() to return true, or b) do this in such a way as to prevent blocking?

Thanks in advance, James

Answer 1

How to create a Java non-blocking InputStream

You can't. Your question embodies a contradiciton in terms. Streams in Java are blocking. There is therefore no such thing as a 'non-blocking InputStream'.

Reader.ready() returns true when data can be read without blocking. Period. InputStreams and Readers are blocking. Period. Everything here is working as designed. If you want more concurrency with these APIs you will have to use multiple threads. Or Socket.setSoTimeout() and its near relation in HttpURLConnection.

Answer 2

For nonblocking IO don't use InputStream and Reader (or OutputStream/Writer), but use the java.nio.* classes, in this case a SocketChannel (and additional a CharsetDecoder).

---

Edit: as an answer to your comment:

Specifically looking for how to create a socket channel to an https url.

Sockets (and also SocketChannels) work on the transport layer (TCP), one (or two) level(s) below application layer protocols like HTTP. So you can't create a socket channel to an https url.

You would instead have to open a Socket-Channel to the right server and the right port (443 if nothing else given in the URI), create an SSLEngine (in javax.net.ssl) in client mode, then read data from the channel, feeding it to the SSL engine and the other way around, and send/get the right HTTP protocol lines to/from your SSLEngine, always checking the return values to know how many bytes were in fact processed and what would be the next step to take.

This is quite complicated (I did it once), and you don't really want to do this if you are not implementing a server with lots of clients connected at the same time (where you can't have a single thread for each connection). Instead, stay with your blocking InputStream which reads from your URLConnection, and put it simply in a spare thread which does not hinder the rest of your application.

Answer 3

You can use the Java NIO library which provides non-blocking I/O capabilities. Take a look at this article for details and sample code: http://www.drdobbs.com/java/184406242.

Answer 4

There is no HTTP/HTTPS implementation using Channels. There is no way to read the inputstream from a httpurlconnaction in a non-blocking way. You either have to use a third party lib or implement http over SocketChannel yourself.

Answer 5

import java.io.InputStream;
import java.util.Arrays;

/**
 * This code demonstrates non blocking read from standard input using separate
 * thread for reading.
 */
public class NonBlockingRead {

    // Holder for temporary store of read(InputStream is) value
    private static String threadValue = "";

    public static void main(String[] args) throws InterruptedException {

        NonBlockingRead test = new NonBlockingRead();

        while (true) {
            String tmp = test.read(System.in, 100);
            if (tmp.length() > 0)
                System.out.println(tmp);
            Thread.sleep(1000);
        }
    }

    /**
     * Non blocking read from input stream using controlled thread
     * 
     * @param is
     *            — InputStream to read
     * @param timeout
     *            — timeout, should not be less that 10
     * @return
     */
    String read(final InputStream is, int timeout) {

        // Start reading bytes from stream in separate thread
        Thread thread = new Thread() {

            public void run() {
                byte[] buffer = new byte[1024]; // read buffer
                byte[] readBytes = new byte[0]; // holder of actually read bytes
                try {
                    Thread.sleep(5);
                    // Read available bytes from stream
                    int size = is.read(buffer);
                    if (size > 0)
                        readBytes = Arrays.copyOf(buffer, size);
                    // and save read value in static variable
                    setValue(new String(readBytes, "UTF-8"));
                } catch (Exception e) {
                    System.err.println("Error reading input stream\nStack trace:\n" + e.getStackTrace());
                }
            }
        };
        thread.start(); // Start thread
        try {
            thread.join(timeout); // and join it with specified timeout
        } catch (InterruptedException e) {
            System.err.println("Data were note read in " + timeout + " ms");
        }
        return getValue();

    }

    private synchronized void setValue(String value) {
        threadValue = value;
    }

    private synchronized String getValue() {
        String tmp = new String(threadValue);
        setValue("");
        return tmp;
    }

}