0

First, I'm not a developer (and I've been coding only for 2 weeks), so feel free to tell me I'm completely misunderstanding the thing (also, I wrote all of this for myself, so I'm sure it's super not cool) :). I want to learn and get it right, so I'm keen to listen to suggestions or complete rewrites.

I want to connect to a socket in non-blocking mode (I'm the client, not the server). I'll mainly need to read from it, but sometimes I'll need to write to it, too. The procedure is as follows:

  • Connect to socket
  • Send some initial requests to login to the server
  • Read from the socket
  • Sometimes, write some stuff (subscribe to certain information, for example)

My solution is as follows (I'm writing it in Java, because I've read it's a fast and good programming language, but I'm happy to change if required... hopefully not needed though!):

public class SocketClient {

    public static void main(String[] args) {
        new Feed().init();
    }

    private boolean isSocketConnected() {
        return socket != null && socket.isConnected();
    }

    public void init() {
        try {
            if (isSocketConnected()) {
                // What here if I'm in non-blocking mode?
                // Would be good to know if the "close API" request succeeded
                // otherwise next time I won't be able to connect to their socket...

                sendCloseRequestToApi();
                socket.close();
            }

            run();
        } catch (Exception e) {
            if (isSocketConnected()) {
                // Same question as above...
                sendCloseRequestsToApi();
                socket.close();
            }
        }
    }

    public void run() throws IOException {
        System.out.println("Starting connection in blocking mode...");

        SocketChannel channel = SocketChannel.open();
        socket = channel.socket();
        socket.setReceiveBufferSize(RECEIVE_BUFFER_SIZE);
        socket.setSendBufferSize(SEND_BUFFER_SIZE);
        channel.connect(new InetSocketAddress("127.0.0.1", 2121));
        channel.finishConnect();

        System.out.println("Finished connecting in blocking mode");

        // Writes to the socket (user and password)
        initialiseTheApi();

        System.out.println("Sent API requests in blocking mode");
        System.out.println("Now we should probably go non-blocking (I guess)");

        channel.register(selector, SelectionKey.OP_WRITE | SelectionKey.OP_READ);
        selector = Selector.open();
        channel.configureBlocking(false);

        System.out.println("Selector created and switched to non-blocking mode...");

        long timeWithoutData = 0;
        boolean needsReconnection = false;
        while (!needsReconnection) {
            selector.select();
            Iterator < SelectionKey > keys = selector.selectedKeys().iterator();

            while (keys.hasNext()) {
                SelectionKey key = keys.next();
                keys.remove();

                if (!key.isValid()) {
                    continue;
                }

                if (key.isWritable()) {
                    // Execute write...
                    // What if I need to know the result to the write operation?
                }

                if (key.isReadable()) {
                    int dataRead = readDataFromSocket(buffer);

                    buffer.flip();
                    if (buffer.remaining() > 0) {
                        // I process the data read here,
                        // but sometimes the data sent is
                        // "reconnect to API". So I need to close
                        // the connection and start again.

                        // How can I do that if I'm in non-blocking mode?
                        // I mean, I need to make sure when I send that request
                        // (for reconnection).
                        // I need to know that the request got to the server and
                        // was processed OK before moving on and
                        // reading/writing again...
                    }

                    if (dataRead > -1) {
                        timeWithoutData = 0;
                    } else {
                        if (timeWithoutData > 0) {
                            long diffInMillis = System.currentTimeMillis() - timeWithoutData;
                            if (diffInMillis > 2000) {
                                System.out.println("Timeout or something? I need to reconnect I think");
                                needsReconnection = true;
                            }
                        } else {
                            timeWithoutData = System.currentTimeMillis();
                        }
                    }

                    // Do I even need this? Already did it before, right?
                    key.interestOps(SelectionKey.OP_READ | SelectionKey.OP_WRITE);
                }
            }
        }

        if (needsReconnection) {
            // We need full reconnection, go back up and reconnect
            init();
        }
    }
}

I removed imports and other non-useful methods for convenience, and to keep the post short.

As you can see in my questions in the code (plus some added ones):

  • Reconnection: If I'm in non-blocking mode, how do I know that my request got sent successfully to the server
  • If I read from the socket and the message is "Reconnect to API", how can I make sure that happens before any other read / write?
  • Do I need to send the interestedOps over and over again?
  • I should only connect once to the socket. The fact that I'm non-blocking doesn't change that, right?

I've seen this could all be simplified using Netty or something, but I'm already bloated with so much stuff! :(

I hope my questions are clear. Let me know otherwise, please.

Thanks a lot.

Will
  • 181
  • 1
  • 14
  • This is a high level explanation about blocking and non-blocking sockets. Hope it helps someway: https://www.scottklement.com/rpg/socktut/nonblocking.html – VeryNiceArgumentException Dec 03 '16 at 19:10
  • 1
    @RaphaelMoita thanks a lot for that link. but I've already seen it. I does say there "Handling many sockets at once using select()", but I think that's not entirely true, is it? I think the good thing about select is to be able to read data quicker than creating a "spinning loop" or a sleep. Instead select will just "notify you" when there's data. So using select is good even if you're the client, and using one socket connection. Am I wrong? – Will Dec 03 '16 at 19:16
  • Only two weeks? You are progressing very quickly. Anyway, you could use an AsynchronousSocketChannel. It's a higher level abstraction and makes working with non blocking connections a lot easier once you get the hang of it http://www.java2s.com/Tutorials/Java/Java_Network/0080__Java_Network_Asynchronous_Socket_Channels.htm – Tesseract Dec 03 '16 at 19:20
  • @SpiderPig two weeks, yes. Thanks for that link. I'll have a look at it and come back with doubts, surely :P – Will Dec 03 '16 at 19:24
  • That's interesting, so with `AsynchronousSocketChannel` when I write I could just do `myFuture.get()` and that'll be the result of the call to the socket server? Hmmm... Ok, but why then it uses in that example the `isRead` variable if you can read and write all at the same time? I may be missing something? Thanks! – Will Dec 03 '16 at 19:29
  • 1
    @SpiderPig do you think I could be using a blocking socket instead? The problem is that socket gives millions of records per second, so reads have to be very very fast, and writes shouldn't really block for a long time, otherwise I'll need to catch up with the previous data sent. Do you think Java is OK for this amount of data? I've seen C, but that looks like Matrix to me (even more than Java!) – Will Dec 03 '16 at 19:31
  • You mean you need to make millions of connections per second? I doubt your computer and internet connection would be able to handle more than a few thousand. Or maybe you want to read those millions of records over the same connection? That could work. Can you give more details/numbers? – Tesseract Dec 03 '16 at 19:42
  • You can use several threads and use blocking connections in each one. – Tesseract Dec 03 '16 at 19:43
  • Thanks for your reply @SpiderPig. I want to read millions of records. It's just one connection from my side. I'm the client, not the server. Also, with my credentials, I cannot login to the socket multiple times, I can only do it once, so multiple threads connected wouldn't work, would it? – Will Dec 03 '16 at 19:47
  • If it's just one connection you only need one thread and you can use a blocking socket. Non blocking sockets are really only useful if you need to make many connections at the same time without having to deal with the overhead that comes from running many threads. – Tesseract Dec 03 '16 at 19:52
  • Ok, then I still have a question... I want to read as fast as possible from the socket, right? But if I read and there's nothing to read at a particular point in time, I want to try again for certain period of time. How can I do that? If I do it with a `Thread.sleep(10)`, I've read that's not efficient nor good, and I should use `Selector` class that Java offers. But that's only available if I'm non-blocking I think? Would you just sleep for X milliseconds instead? See this: http://stackoverflow.com/questions/3125645/why-use-select-instead-of-sleep – Will Dec 03 '16 at 19:58
  • Blocking sockets automatically take care of those things for you. If you try to read from them your program will wait (blocking just means to wait) until some data has been received and then continue immediately. So you will be able to download data at the maximum speed. Reading into a ByteBuffer with a SocketChannel is very efficient. You don't need to do take any extra measures. – Tesseract Dec 03 '16 at 20:04
  • btw. you don't need to call `finishConnect` for blocking sockets. `connect` already blocks until the connection is established. – Tesseract Dec 03 '16 at 20:08
  • 1
    @SpiderPig you're my hero! Just last question then: ok, so I can remove the `setBlocking` call I make, so the read and writes will block, ok cool. And then how can I make it time out? Can't see any parameter I can send to `read`, but I can read in the socket I can do: `public synchronized void setSoTimeout(int timeout) throws SocketException {`. Would that work you think? PD: I need a developer in my life, there's so many things I don't know about all of this :( – Will Dec 03 '16 at 20:11
  • That should work. You can also search stackoverflow for more information on that e.g. http://stackoverflow.com/questions/17340970/read-timeout-for-an-nio-socketchannel – Tesseract Dec 03 '16 at 20:18
  • 1
    @SpiderPig thank you very very much man! You've been great help. I was trying to do something that just didn't make sense. In my case I can definitely use a blocking connection, which I just didn't know about :/. Internet is a bad source of information sometimes! I kept reading over here not to use a blocking connection :D. But now it makes perfect sense the different scenarios. – Will Dec 03 '16 at 20:25
  • One more thing - you may want to use the `Socket` class and download your data with an `InputStream` instead of using a `SocketChannel`. Because according to this post http://stackoverflow.com/questions/2866557/timeout-for-socketchannel-doesnt-work `SocketChannel`s don't work well with `setSoTimeout`. – Tesseract Dec 03 '16 at 20:27
  • Just make sure to pass an array into the `InputStream.read` method to read in a large number of bytes at once. – Tesseract Dec 03 '16 at 20:30
  • Oh no! Ok I'll have a read around. Maybe now in 2016 it works :/ Thanks I'll let you know. – Will Dec 03 '16 at 20:32
  • Now I have problems :(. `Be careful if performance is a concern. Using Channels.newChannel is not the same as using the SocketChannel because data is copied from the input stream, not directly from the file descriptor in native code. This means that the benefit of using direct byte buffers with socket channels is lost.` (in the comments of this article: https://technfun.wordpress.com/2009/01/29/networking-in-java-non-blocking-nio-blocking-nio-and-io/). So: I can't use a timeout, so I have to use a selector, but if I want a selector I have to be non-blocking :(. Any alternatives then? – Will Dec 03 '16 at 20:44
  • Looks like I'm out of luck? http://stackoverflow.com/questions/15120369/how-does-one-implement-a-timeout-in-blocking-mode-nio – Will Dec 03 '16 at 20:51
  • As I said, use a simple `Socket`, call it's `getInputStream` method and read from that with `InputStream.read(array)`. Then timeouts will work fine. No need to deal with channels at all. Also you are too worried about performance. I doubt your internet connection speed is in the GBit range. It will be more than fast enough either way. Here is an example http://www.rgagnon.com/javadetails/java-0542.html – Tesseract Dec 03 '16 at 20:51

1 Answers1

0

I was trying to do something that just didn't make sense. In my case I can definitely use a blocking connection, which I just didn't know about :/. Internet is a bad source of information sometimes! I kept reading over here not to use a blocking connection :D. But now it makes perfect sense the different scenarios. – Will

Armali
  • 18,255
  • 14
  • 57
  • 171