Size of event bus in vert.x

Question

I am using vert.x to read a file and transform and then push to kafka. I am using 2 verticles, without using any worker thread (I dont want to change the order of logs in the file).

Verticle 1 : Read the file and filter 
Verticle 2 : Publish to kafka

Each files contain approximately 120000 lines

However, I observed that after sometime i stop observing logs from verticle 1. I am suspecting that event bus is getting full, so Consumer is still consuming, but producer thread is waiting for event bus to get empty.

So My questions are 1. What is the default size of event bus? In Docs it says

    DEFAULT_ACCEPT_BACKLOG
The default accept backlog = 1024

2. How do I confirm my suspicion that publisher thread is blocked?

Answer 1

VertX uses Netty's SingleThreadEventLoop internally for its event bus, maximum pending tasks allowed is Integer.MAX_VALUE which is probably 2 billion messages.

You may have to try VertxOptions.setWarningExceptionTime(long warningExceptionTime) to set the value lower than default (5sec) to see if there is any warning about blocked thread.

Answer 2

To complement @iwat answer, in the version I am using, it looks like the max size is read from a system property:

protected static final int DEFAULT_MAX_PENDING_TASKS = Math.max(16, SystemPropertyUtil.getInt("io.netty.eventLoop.maxPendingTasks", 2147483647));

So you can control the size of the queues in front of the Verticles by setting that system property.

If the event bus is full (the queue in NioEventLoop reaches the max size), the task will be rejected. So if you hit that, you should start to see error responses to your messages, you should not see any blocked producers.

Answer 3

I'm not sure the accept-backlog setting has any effect on the eventbus, given the documentation it might have something to do with the netserver, but from a short scan of the code I haven't found any use in the eventbus.

The event bus however does deliver the message immediately, messages don't get queued up somewhere (at least that's what I understand from the code). So regarding your first question, it doesn't have any size, at least not when running locally (don't know about the clustered version, but I assume that doesn't apply in your case anyway)

To confirm an (eventloop) thread is actually blocked is easy, there should be tons of exceptions in your log stating the event loop is blocked.

I guess your problem is somewhere else, but that's actually hard to tell without any code or meaningful logs.