26

I am in the process of choosing a networking library to implement a client/server system that cannot spare any microsecond. It will implement its own protocol to send and receive messages. I am looking for a good NIO framework that will allow me to easily develop the server and the client, without having to worry too much about the low level selector details. Everyone recommends me Netty but I would like to experiment with 2 or 3 other alternatives before committing my team with a framework. One thing I did not like very much about Netty is how it handles ByteBuffers with its own ByteBuf implementation and reference counting. Can anyone share your thoughts and alternatives?

chrisapotek
  • 6,007
  • 14
  • 51
  • 85
  • 2
    Recommendation questions are off-topic, bounty or no bounty. – user207421 May 26 '14 at 23:31
  • @EJM I just hit 3k, and it says "this question has an open bounty and cannot be closed." It also prevents me from flagging... – Anubian Noob May 27 '14 at 14:50
  • Both Netty and Mina use their own ByteBuffer abstractions. Mina 3 removes this abstraction but it is still a work in progress. Netty drives JBOSS which is a much more complex implementation of a reactor framework. Mina is more simple and low level. – bond May 27 '14 at 22:31
  • @AnubianNoob I'm aware that questions with open bounties cannot be closed. That's why I posted the comment. – user207421 May 28 '14 at 22:51

3 Answers3

29

We have developed a NIO networking library that performs under 2 microseconds over loopback without producing any garbage for the GC. As Peter Lawrey mentioned, the native JDK selector produces a lot of garbage but we have fixed all these garbage leaks by implementing our own epoll selector. Busy waiting the selector thread is great for latency but there must be a balance not to burn the chip or consume a lot of energy. Our selector implementation use low-level tricks to implement a kind of energy saving mode that takes care of that balance.

Besides CoralReactor, you can also take a look on Grizzly and Mina, but we haven't played with these frameworks yet.

For some Netty TCP performance benchmarks you can take a look here.

rdalmeida
  • 1,806
  • 12
  • 16
  • I am interested in this; however the Native JDK only produces ~150 MB of garbage every 30 or so million requests. At 118 MB/s that is every 2 minutes give or take. I would like to see benchmarks against JDK NIO instead of Netty. – bond May 27 '14 at 22:26
  • the only part of the JDK NIO that creates garbage relates to the ArrayList/List mechanism it stores when moving the select events to and from the Selector. This amount of garbage is relatively little. Netty and Mina then add an absurd amount of garbage on top of it. JDK NIO also does a bunch of horrible things with the synchronized keyword but thats another discussion. I am all for a better NIO than the JDK version; I just want to see the numbers of the NIO itself. – bond May 28 '14 at 01:03
  • 33
    Not free, not open source, no clear licensing. – Norswap Feb 11 '17 at 15:54
  • The benchmark should use one unique client. – Amir Fo Dec 10 '19 at 12:23
23

This is assuming you really want to save every micro-second. Most applications don't have such strict requirements.

If you want to save micro-seconds, you will want to use busy waiting non-blocking NIO for threads on dedicated cpus. This doesn't scale well as you need to have plenty of CPU but does minimise the latency for handling IO. I suggest you also bind the isolated CPUs to minimise jitter.

You will want to avoid using Selectors as they block and/or create quite a bit of garbage adding to GC pauses.

Also to minimise latency you will want to use a low latency, kernel bypass network adapter such as Solarflare.

You will want to use a push parser so long messages can be decoded/parsed as they download. i.e. you won't want to wait until the whole messages is received before starting.

Using these tricks in combination can save 10 - 30 micro-seconds off every request or inbound event.

Netty is a better solution for scalability ie, higher net throughput, but at a small cost to latency, as do most frameworks which are based on support web services where milli-seconds delays are tolerable.

Optimus Prime
  • 6,817
  • 5
  • 32
  • 60
Peter Lawrey
  • 525,659
  • 79
  • 751
  • 1,130
15

If you are okay with using at least some Scala, Spray is a great alternative to Netty. On the long run, the Play framework is for example intending to migrate from Netty to Spray. Spray offers different levels of TCP abstractions. Those are:

  1. Chunk level
  2. Request level (HttpRequest / HttpResponse)
  3. Marshalled object level

The deeper you dig down into the stack, the more raw the delivered information is. In the chunk level API, you come pretty close to original byte buffers. I never used this low abstraction level myself, but I heard good things.

Spray builds on top of Akka IO which is again built on top of Java NIO. All functionality wraps around Actor abstractions what makes it easy to build parallel applications using Spray. I think a chat server would be a perfect use case. Since Akka offers a Java API, you should be able to use Spray with mostly this API. However, you will probably need to read some Scala sources every now and then. Eventually, Spray will merge completely into Akka.

Edit: Quote from the Spray web site: "Spray is no longer maintained and has been superseded by Akka HTTP. Playframework started experimentally supporting Akka HTTP Server backend starting from Play 2.4.X. In Play 2.6.X versions, play completely migrated to Akka HTTP server backend.

yerlilbilgin
  • 3,041
  • 2
  • 26
  • 21
Rafael Winterhalter
  • 42,759
  • 13
  • 108
  • 192