slow java socket performance on localhost

Question

I am building a simple java network app and I have a performance issue.
The app is very simple and contains 2 parts (source code):

The server which is the main thread.
The client which is new thread that has started by main.

I am trying to send a lot of small data but the maximum send rate is 50 package per second.
So In my search I have found this question with the very same problem.
When I added the Buffered stream to the output stream, the rate goes bigger (thousand per second)
In wireshark I can see the difference very nicely:

In the slow run there is push flag and in different package the ack flag, and some ack can reach even 45 millisecond of delay, and the cpu use was very low.
In the fast run all the package contains push and ack together and for all of then the delay is no more then 1 millisecond, and the cpu use was high.

I found also this related question that does not help.

Why is that simple line of code make so different?
And why the simple solution in java so slow?

From the comment under the [linked question](https://stackoverflow.com/a/4451512/4417306) "*I can explain that. If you use a stream that doesn't use a buffer, then each write results in a syscall. The overhead can be thousands of extra machine instructions per syscall*" — Mark, Dec 06 '18 at 19:28
But then the cpu will be high and it is not the case here (the cpu is low in the slow run), furthermore I flush the stream after every write and this should also do syscall. — amit, Dec 06 '18 at 19:42
Have you been using other resources for the same localhost? are you using JDBC to request querys? — Jose Antonio, Dec 06 '18 at 20:42
No. this is the only running process (except of course the operation system process) — amit, Dec 07 '18 at 09:22

rustyx · Answer 1 · 2018-12-06T20:27:20.717

Sockets are historically opened using the Nagle's algorithm enabled, which delays small packets in an attempt to optimize network communication.

When working with small packets you should set the TCP_NODELAY socket option. In Java you can do it by calling socket.setTcpNoDelay(true);

After the adjustment I'm getting:

12000 packets/s without a buffer
26000 packets/s with a buffer

My adjusted code:

public class Test {
    static DataOutputStream cout(Socket s) throws IOException {
        return new DataOutputStream( (s.getOutputStream()));
//        return new DataOutputStream(new BufferedOutputStream(s.getOutputStream()));
    }
    static DataInputStream cin(Socket s) throws IOException {
        return new DataInputStream( (s.getInputStream()));
    }
    public static void main(String[] args) throws IOException, InterruptedException {
        ServerSocket server = new ServerSocket(12345);
        new Thread(() -> {
            try {
                Thread.sleep(1000);
                Socket client = new Socket("127.0.0.1", 12345);
                client.setTcpNoDelay(true);
                DataOutputStream out = cout(client);
                DataInputStream in = cin(client);
                long tm1 = System.currentTimeMillis();
                int lastCount = 0;
                for (int i=0;i<300000;i++) {
                    int a = in.readInt();
                    out.writeInt(a);
                    out.flush();
                    long tm2 = System.currentTimeMillis();
                    if ((tm2 - tm1) >= 1000) {
                        System.out.println(i - lastCount);
                        lastCount = i;
                        tm1 += 1000;
                    }
                }
            } catch (IOException | InterruptedException e) {
                e.printStackTrace();
            }
        }).start();
        Socket client=server.accept();
        client.setTcpNoDelay(true);
        DataOutputStream out = cout(client);
        DataInputStream in = cin(client);
        for (int i=0;i<300000;i++){
            out.writeInt(i);
            out.flush();
            if (i!=in.readInt()){
                System.out.println("ERROR");
            }
        }
        client.close();
        server.close();
        Thread.sleep(1000);
    }
}

slow java socket performance on localhost

1 Answers1