Hmm, IO::Async can't even handle 10k connections?

Question

Hi I built a very simple HTTP server and only to do a stress testing on POE module.

Here's how I tested it:

for x in {1..10000}; do
  curl xxxxx:12342 --retry-delay 5 --retry 3 -d randombytes$x 
done

When I do that I see multiple "Connection Reset" by peer message, so for this 10k connection I lost about 1200 requests in total.

Is there anyway to improve its performance? Tested nginx on the same server, no kernel parameters was changed during that, no connection reset message at all.

EDIT

I already increased max fileno / per user max fileno, the former was fs.file-max = 3246455, and the latter was 10240

Attached source code:

use strict;
use warnings;
use feature 'say';

use Net::Async::HTTP::Server;
use IO::Async::Loop;

use HTTP::Request;
use HTTP::Response;
use Data::Dumper;

my $loop = IO::Async::Loop->new();

my $httpserver = Net::Async::HTTP::Server->new(
    on_request => sub {
        my ($self, $req) = @_;
        my ($code, $result) = (200, "It works\n");

        my $response = HTTP::Response->new ($code);
        $response->add_content ($result);
        $response->content_type ("application/json");
        $response->content_length (length $response->content);

        $req->respond($response);
    },
);

$loop->add( $httpserver );

$httpserver->listen(
    addr => { family => "inet", socktype => "stream", port => 8080 },
    on_listen_error => sub { die "Cannot listen - $_[-1]\n" },
);

say 'Listening normally on ', 8080;

$loop->run;

Tags say POE, description and code say IO::Async. I'll retag — LeoNerd, Apr 20 '14 at 12:55

score 5 · Answer 1 · answered Apr 20 '14 at 12:55

5

You'll probably find you're coming up on the "file descriptors per process" limit. By default that's 1024.

$ ulimit -n
1024

If you want to handle 10,000 concurrent connections then obviously you'll need at least one file descriptor per connection. Accounting for a few extra for the process and some extra headroom for stale ones, you probably want to set this to about 11k.

Only root can increase the FD limit:

$ sudo -s
Password:
# ulimit -n 11000
# ulimit -n
11000

At this point you can now run your server. (Don't forget to switch back to the UID required, so as not to run it as root).

When running such a large number of connections via IO::Async you'll probably want to use something better than the default built-in poll or select-based Loops. For example on Linux you likely want to install the epoll loop, as well as the linux one so it will automatically load.

$ cpan IO::Async::Loop::Epoll
$ cpan IO::Async::Loop::linux

answered Apr 20 '14 at 12:55

LeoNerd

8,344
1
29
36

ulimit shows 10240 (which is adjusted prior to this stress testing), I increased it to 65535 anyway – daisy Apr 20 '14 at 13:04
Also, how can I know if these two modules are "selected"? I have them installed – daisy Apr 20 '14 at 13:05
my $loop = IO::Async::Loop->new; say ref($loop) # Will print the kind of loop you actually have – LeoNerd Apr 20 '14 at 13:50
As to knowing how many spare FDs are left, I'm not sure offhand. I suspect that's a topic for a new SO question. – LeoNerd Apr 20 '14 at 13:52
Is there a "backlog" option in IO::Async? – daisy Apr 20 '14 at 14:44
Resolved. It was the `queuesize` in the listen method. – daisy Apr 20 '14 at 14:56
@warl0ck, Please write that up as an answer. – ikegami Apr 20 '14 at 15:18

Hmm, IO::Async can't even handle 10k connections?

1 Answers1