0

I have a question about netty server monitoring. I want to see how many open connections to the server are there, so to do that we have a code that has an atomic integer counter that is increased by 1 on ChannelInitializer.initChannel(..) method and decremented by 1 in closeFuture listener for that SocketChannel channel. For some reason that I can't figure out, it doesn't go to zero and stays positive under heavy load. I was hoping that maybe there is some better way of tracking these open channels?

  @Override
  protected void initChannel(SocketChannel channel) throws Exception {
          currentConnections.incrementAndGet();
          channel.closeFuture().addListener(f -> currentConnections.decrementAndGet());
  }

UPDATE: So the number stays positive after clients stop sending traffic and disconnect.

alobodzk
  • 1,284
  • 2
  • 15
  • 27
  • 1
    Doesn't go to zero under heavy load? Why would you expect there to be no connections if there's a heavy load? – Kayaman Oct 25 '19 at 10:42
  • True, what I should say is that after heavy burst of messages, when tests stop sending new ones and clients are terminated I would expect to see 0. – alobodzk Oct 25 '19 at 10:46
  • It's (most likely) a TCP/IP stack related [thing](https://stackoverflow.com/questions/3757289/tcp-option-so-linger-zero-when-its-required). Basically the connection isn't closed immediately, but it "lingers" for a while. It's normal, and unless it's causing you problems (running out of resources), there's no need to touch it. – Kayaman Oct 25 '19 at 10:48
  • That would be a good explanation, but the number seem to never go down. – alobodzk Oct 25 '19 at 11:19
  • Well is there a reason why you need it to go down? Why are you tracking it? Do you have a problem, or are you just expecting it to behave in a certain way? – Kayaman Oct 25 '19 at 11:21
  • To understand the state of the system really, monitoring and alerting, how many open connections are seems like a reasonable metric to have. – alobodzk Oct 25 '19 at 15:26
  • It might be, I don't know Netty internals well enough to say. But looks like the maintainers [aren't really sure either](https://github.com/netty/netty/issues/4981), although there's some [recent interest](https://github.com/netty/netty/pull/8680). – Kayaman Oct 25 '19 at 15:49

1 Answers1

1

One way is to add a ChannelInboundHandler and override the channelRegistered and channelUnregistered like so:

  @Override
  public void channelRegistered(final ChannelHandlerContext ctx) {
    if (ctx.channel().remoteAddress() != null) {
      connections.inc();
    }
  }

  @Override
  public void channelUnregistered(final ChannelHandlerContext ctx) {
    currentConnections.dec();
  }

Another way is to only add the channel to a ChannelGroup when it's registered, and use ChannelGroup#size when you want to query the current connection count.

I guess the former incurs less overhead, but the later allows you to performs actions on all active channels if you need this sort of functionality anyway.

Eran Harel
  • 2,325
  • 19
  • 28
  • Cheers, what would be the advantage of using channelRegistered/Unregistered over channelActive/Inactive? – alobodzk Oct 28 '19 at 11:34
  • Also should that remoteAddress check be also present in the channelUnregistered method so the counter wouldn't go into negative range? Or there is something going on that it's not required? – alobodzk Oct 28 '19 at 11:44
  • Strictly speaking, according to the netty-in-action book I think it is more "correct" to use the active/inactive states if you want to monitor connected clients. I use the registered/unregistered as it represents the entire channel life-cycle which represents the resources you're holding more correctly. I guess both events will work for you – Eran Harel Oct 29 '19 at 09:10
  • The channel that lacks the remote address is the parent channel - it's not a representation of one of the inbound "connections". Maybe Norman Maurer can explain this better... – Eran Harel Oct 29 '19 at 09:13