3

I'm new at network programming and i have been searching for a solution to my problem here but couldn't find one. What I want is to have a server that can receive files from multiple sockets at the same time. When a server accepts new connection socket it wraps that socket with a ClientThread class. Here is the code:

public class Server extends Thread {
    private ServerSocket server;
    private Vector<ClientThread> clients;

    @Override
    public void run() {
        listen();
    }

    private void listen() {

    new Thread("Listening Thread") {

        @Override
        public void run() {
            while (true) {
                try {
                    Socket socket = server.accept();

                    ClientThread newClient = new ClientThread(socket);
                    newClient.start();
                    clients.addElement(newClient);

                } catch (IOException | ClassNotFoundException e) {
                    e.printStackTrace();
                }
            }
        }
    }.start();
}

ClientThread is a private class inside the Server class. It's always listening for an Object from ObjectInputStream, but also I want to be able to receive one big file after the object. And that is why I think i should use multithreading. Here is the code:

private class ClientThread extends Thread {

    public Socket socket;
    private boolean loggedIn;
    private ObjectInputStream ois;
    private BufferedInputStream bis;

    public ClientThread(Socket socket) {
        this.socket = socket;
        loggedIn = true;

        InputStream is = socket.getInputStream();
        ois = new ObjectInputStream(is);
        bis = new BufferedInputStream(is);
    }

    @Override
    public void run() {
        receive();
    }

    private void receive() {

        while (loggedIn) {
            try {
                // this method blocks i guess
                Object object = ois.readObject();

                // after the object comes the large file
                byte[] bytes = new byte[SOME_SIZE];

                int bytesRead;
                int totalRead = 0;

                // reading the large file into memory
                while ((bytesRead = bis.read(bytes, totalRead, bytes.length - totalRead)) > -1) {
                    totalRead += bytesRead;
                }

                // rest of the code for handling received bytes.......

            } catch (ClassNotFoundException | IOException e) {
                e.printStackTrace();
            }
        }
    }

}

I'm not sure if receiving data like this is even possible since all these client sockets are sending data to the same port on this server (i guess?). And if clients are sending data at the same time, Server needs to know which data is for which client. Is this already taken care of, or i need entirely different approach here?

I don't know if this is a stupid question, but like I said I'm just starting learning this stuff. Also i couldn't test my program because i don't even have code for the Client yet. Just want to make sure I don't go wrong at the very start. If this is wrong, feel free to post some ideas. :) Thanks!

Matthieu
  • 2,736
  • 4
  • 57
  • 87
pavlee
  • 828
  • 1
  • 12
  • 19
  • Last year I add a similar request and I ended up using Netty http://netty.io/, and they have a wiki with several examples http://netty.io/wiki/, for sockets check telnet or maybe udp. – Deceiver Aug 12 '16 at 11:56
  • 1
    "I'm not sure if receiving data like this is even possible since all these client sockets are sending data to the same port on this server (i guess?)" - Clients connect to one single port, yes. But they are given a unique **local** port. So yes, you can in fact receive from many clients connected to the same (public server-) port concurrently. – Fildor Aug 12 '16 at 11:59
  • 2
    Well, to be clear, they are given a unique local port if they are all in the same client host, which is the only time a unique local port would be necessary. – user207421 Aug 12 '16 at 12:23
  • [That other answer](http://stackoverflow.com/a/14388707/1098603) is worth reading if you want to understand more about IP:port uniquenesses. – Matthieu Aug 12 '16 at 12:40
  • Thanks all for answers! :D – pavlee Aug 12 '16 at 14:32

2 Answers2

3

For a start it's not bad :) You can improve later on by using a Selector but that's another topic.

Some clarifications though: the ServerSocket listens on a specific port. When a remote client connects to it, a communication channel (i.e. socket) is created. If another client connects, another socket is created. Both sockets are different channels and won't interfere with each other because they are connected to a different remote IP and port.

It all has to do with how TCP headers and IP headers are formed: a TCP data packet is sent with its header containing the source and destination port, on top of IP header containing the source and destination IP. Those are used to discriminate between the different sockets.


Regarding the "broadcast" you want to do (as per your comment in @Rajesh's answer), you have options:

  • Do it yourself in pure TCP with ServerSocket and Socket like you started
  • Switch to UDP and use MulticastSocket, which has the advantage of issueing a single send, but you'll have to deal with missing/unordered datagrams in your client code (UDP does not guarantee delivery or ordering, like TCP does)
  • Check NIO with Selector and SocketChannel
  • Investigate frameworks like jGroups or Netty which do the I/O stuff for you

As you're learning, I suggest you do that in the above order. Using a framework is nice, but going through coding yourself will teach you a lot more.

Anatoly Shamov
  • 2,608
  • 1
  • 17
  • 27
Matthieu
  • 2,736
  • 4
  • 57
  • 87
  • Thank you for your answer! It all makes sense now that i read this. Glad this could work. :) – pavlee Aug 12 '16 at 14:29
  • @pavlee, network programming will now offer you a whole new world of possibilities, enjoy! :) – Matthieu Aug 12 '16 at 17:03
  • Thanks. I won't use frameworks for now because I want to understand how it works in detail. Guess I should be learning more about it before implementing. – pavlee Aug 13 '16 at 21:52
  • @pavlee I think you're right. NIO is also worth checking once you understand the basics. Also, if you like answers, don't hesitate to upvote them ;) – Matthieu Aug 14 '16 at 04:00
  • Sorry, I tried to upvote but I don't have enough reputation to do that yet. I'll do it when I get it. This is my first question. :) – pavlee Aug 14 '16 at 08:41
  • @pavlee if you want to quickly check NIO I have written a [documentation](http://stackoverflow.com/documentation/java/5513/nio-networking) with sample code you can try out-of-the-box. – Matthieu Aug 17 '16 at 07:49
  • Great example and easy to understand. By the way I already started experimenting with NIO and Selector. – pavlee Aug 17 '16 at 21:02
  • In your opinion, would implementing the second option be too hard for a beginner? In pure TCP i would need to send to every client separately, which i guess would split my upload speed by the number of clients. – pavlee Aug 20 '16 at 21:36
  • 1
    @pavlee UDP is easier than TCP because it's "fire & forget". It depends how many clients you have, how critical are the data and the type of network (LAN or Internet, "lengthier" networks like Internet are more likely to drop UDP datagrams). The difficult part is dealing with unordered or lost datagrams. – Matthieu Aug 21 '16 at 04:20
1

This will work functionally. Each thread is reading from a separate socket connected to different client (address + port). They are separate streams, so no issues in reading from that like this.

However it would be much better to use asynchronous sockets.

Few things that can be taken care in the current implementation:

1) As a good practice, close the streams/sockets when transfer is complete.

2) For every new connection, a new thread is created. That will not scale. Even some one can send many requests and bring down your app. Would be better to use a thread pool. "ClientThread" can just implement "Runnable" and when a new connection is received, just submit the new "ClientThread" to thread pool. (In this case, would be better to name it as ClientTask instead of ClientThread)

As mentioned, it would be much more efficient and scalable to use asynchronous socket, but it will take some time master it. With this, you can use just one thread to read all sockets in parallel and depending on load, can use the same thread or a pool of threads to process the data received from all the sockets. Note that, even if use a pool, you will not need separate thread for processing each socket...Just to make best use of multiple CPU Cores, can use multiple threads to process the data.

You may try either java nio (Selector + SocketChannels) or netty library. Netty is much easier to use compared to nio.

Rajesh Jose
  • 314
  • 2
  • 12
  • Using a thread pool will just limit the number of connections. Scaling would be even worse. The solution is to use a `Selector` and a thread pool to process "*atomic*" events (connect, read, write, ...). – Matthieu Aug 13 '16 at 08:16
  • Well, I consider throughput and scalability two different things...Consider this - Service gets 1000 connections at once, it will spawn 1000 threads. Straightaway jvm will go down with stack overflow. A thread pool with core size of average number of request will be stable, though throughput will be throttled at the level it is designed for. – Rajesh Jose Aug 13 '16 at 09:06
  • Any system will have max it can handle depending on resources available. Beyond that it has to handle gracefully. If one component in the system consumes more resources unpredictably, others will starve or the system will go down. No doubt asynch socket will do better, but when synch socket is used, it is better to throttle so that it consumes resources in a predictable way. U can scale up/down by allocating more resources and permitting more threads depending on average/max number of parallel requests expected; Beyond what is planned/designed, handle gracefully without system going down. – Rajesh Jose Aug 13 '16 at 09:07
  • @Rajesh What i actually want here is this: When data from one client is received I want the server to broadcast that data to all other connected clients. I need continuous connection for this, that is why I don't close the sockets. To be clear I want server to be a "middle-man" between clients. Is this OK? – pavlee Aug 13 '16 at 09:40
  • From what i see the best would be to use Selector for checking the channels and thread pool for "receiving threads". – pavlee Aug 13 '16 at 09:47
  • 1
    @pavlee Yes, this is a fit case to use asynch sockets. Separate thread for reading from each client will not scale. In above code....Yes, u need to keep skt open as long as u need to communicate. But at some point it will come out? Either an exception or client leaves gracefully. At that point u need to close else it can cause resource leak. – Rajesh Jose Aug 13 '16 at 10:37
  • 1
    @pavlee Couple of other points - 1st point - In above code, reading stream socket's input stream will normally return (< 0) when other side closes. So ur while loop for read will come out only after other side closed. If client will send multiple files/records in same connection, u need to include some identifier to detect end of each file/record. 2nd point - not clear how u are writing back to clients. Whether in this code or asynch sockets, make sure u don't end up writing back to same client socket from multiple threads at same time. – Rajesh Jose Aug 13 '16 at 10:40
  • What I meant is a thread pool of depth 5 will allow 5 connections at the same time. The 6th one will be delayed until one of the first 5 disconnects. And it's all about how the software is designed. @pavlee as you're experimenting with network programming, get the basic things running first, then switch to more powerful designs like NIO. – Matthieu Aug 13 '16 at 12:05
  • @pavlee for that type of broadcasting I edited my answer to ginve you some other trails to follow. As you're learning, I think you should go step by step. – Matthieu Aug 13 '16 at 12:35