How to avoid tight thread loop?

Question

I have a Java program with multiple sockets that occasionally have data that need to be read and processed, but there is an indeterminate amount of time which there is no data to be read. I need a good way to constantly check if there is data in the sockets, and process the data. Assigning one thread per socket is not a good idea since there could be too many sockets and use too much memory.

Currently, I have a couple threads, each one assigned to service its own list of sockets. If there was nothing to read in any of the sockets, then sleep one second, then loop. If there was something to read in any of the sockets, just loop without waiting and iterate through the sockets again.

The reason I do this is because I don't want to use up too much resources if there is nothing to read, and the one second delay is not a problem. The only down side is that there is no flexibility for sockets to jump threads, so the worst case scenario is that a single thread is overloaded with work, while the other threads are doing nothing.

Another idea I've had: create a thread pool, and queue up all the sockets to be serviced, and re-add them when they are serviced, but there is no good way to know if none of the sockets need servicing and the threads can take a break to free up CPU cycles.

Is there a good way to assign threads tasks, but not overload computer resources if there is nothing to do?

Ideally an event is triggered each time there is data available in a socket, but as far as I know, there is no way to do this, and I must poll the sockets.

To reiterate, I do not want a one to one relationship between socket and thread.

You seem to be looking for [Selectors](https://docs.oracle.com/javase/7/docs/api/java/nio/channels/Selector.html). — John Bollinger, Apr 13 '16 at 17:51
Check out [this answer](http://stackoverflow.com/a/24617983/2398375) — Vince, Apr 13 '16 at 18:14

score 2 · Answer 1 · answered Apr 13 '16 at 18:00

there could be too many sockets and use too much memory.

You can achieve 1,000 to 10,000 this way. Memory is much cheaper than it was when NIO was introduced 12 years ago and threads are more efficient and scalable than they used to be.

I have a couple threads, each one assigned to service its own list of sockets. If there was nothing to read in any of the sockets, then sleep one second, then loop.

I use a pause which busy waits for a short period and yeilds and finally sleeps for an escalating period of time.

You can use Selectors, but these are not simple to use correctly. In this situation I would use a library like netty or at the very least read the code it uses.

The only down side is that there is no flexibility for sockets to jump threads, so the worst case scenario is that a single thread is overloaded with work, while the other threads are doing nothing.

This is where using a thread per socket is better.

I must poll the sockets.

You can use Selectors, but these are single threaded and switch sockets between selectors is not simple.

I would reconsider using more threads for simplicity.

How to avoid tight thread loop?

1 Answers1