How can I efficiently poll a lot of servers?

Question

I am looking for a good way to poll a lot of servers for their status through TCP. I am currently using synchronous code and the Minecraft Query Protocol, but whenever a server is offline the rest of the queue gets hold up.

Another problem I am experiencing with my current code is that some servers tend to block my server I use for polling in their firewall, and thus their servers appear offline on my serverlist.

I am using a Ruby rake task with an infinite loop in which every Minecraft server in my MongoDB database gets checked and updated every +- 10 minutes (I try to set this interval by letting the loop sleep (600/ s.count.to_i).ceil seconds.

Is there any way I can do this task efficiently (and prevent servers from blacklisting my IP in their firewall), preferably with Async code in Ruby?

score 4 · Accepted Answer · edited May 23 '17 at 11:49

4

You need to use non-blocking sockets to check - multithreading. The best thing to do is spawn several threads at once to check several servers at once - that way your main thread won't get held up.

This question contains a lot of information about multithreading in Ruby - you should be able to spawn multiple concurrent threads at once, or at least use non-blocking sockets.

Another point given by @Lie Ryan, you can use IO.Select to poll a array of servers, all at once. It will return an array of "online" servers when it's done - this could be more elegant than spawning multiple threads.

edited May 23 '17 at 11:49

Community

1
1

answered Feb 26 '13 at 17:10

Christian Stewart

15,217
20
82
139

1

rather than multithreading, it's usually better to use IO.select. – Lie Ryan Feb 26 '13 at 17:15
I like the IO.Select method. Should I just execute the IO.Select line of code every let's say 10 minutes then? – kiwiboom Feb 26 '13 at 17:21
You can do this, make sure the previous check has already ended though – Christian Stewart Feb 26 '13 at 17:23
I'll look into using IO.select, thanks. I'll accept this answer for now – kiwiboom Feb 26 '13 at 17:28
1

[em-synchrony](https://github.com/igrigorik/em-synchrony) makes it a lot easier to write non-blocking asynchronous code using fibers. It's worth checking out rather than doing it with nested callbacks. – tadman Feb 26 '13 at 18:03

How can I efficiently poll a lot of servers?

1 Answers1