104

I had the requirement to build up a REST API in node.js and was looking for a more light-weight framework than express.js which probably avoids the unwanted features and would act like a custom-built framework for building REST APIs. Restify from its intro is recommended for the same case.

Reading Why use restify and not express? seemed like restify is a good choice.

But the surprise came when I tried out both with a load.

I made a sample REST API on Restify and flooded it with 1000 requests per second. Surprise to me the route started not responding after a while. The same app built on express.js handled all.

I am currently applying the load to API via

var FnPush = setInterval(function() {           
    for(i=0;i<1000;i++) 
        SendMsg(makeMsg(i));                
}, 1000);

function SendMsg(msg) {
    var post_data = querystring.stringify(msg);
    var post_options = {
        host: target.host,
        port: target.port,
        path: target.path,
        agent: false,
        method: 'POST',
        headers: {
                'Content-Type': 'application/x-www-form-urlencoded',
                'Content-Length': post_data.length,
                "connection": "close"
            }
    };

    var post_req = http.request(post_options, function(res) {});
    post_req.write(post_data);  
    post_req.on('error', function(e) {          
    }); 
    post_req.end();
}

Does the results I have got seem sensible? And if so is express more efficient than restify in this scenario? Or is there any error in the way I tested them out?

updated in response to comments

behavior of restify

  1. when fed with a load of more than 1000 req.s it stopped processing in just 1 sec receiving till 1015 req.s and then doing nothing. ie. the counter i implemented for counting incoming requests stopped increment after 1015.

  2. when fed with a load of even 100 reqs. per second it received till 1015 and gone non responsive after that.

Community
  • 1
  • 1
Mithun Satheesh
  • 27,240
  • 14
  • 77
  • 101
  • 3
    It is possible that restify somewhere block while parsing routes or request data, and does not do it efficiently, which leads to spikes in response times with high load. Express.js is lightweight but rich in functionality. The way it is made, still makes it light because unused functionality has not much impact on overal performance. As well it is well maintained and used by big companies, one of the examples: MySpace. I can't see any disadvantages of using Express.js for REST API (I actually did exactly that), it actually allows you in a future to improve your API as functionality is there. – moka Jul 11 '13 at 09:42
  • 1
    @Munim: thanks for the graphs. but the page says "**note, this chart is out of date since Restify performance issues were resolved**".. But seems like nothing is resolved.!! – Mithun Satheesh Jul 11 '13 at 10:06
  • 1
    @mithunsatheesh I noticed those too. But since the author didn't conduct fresh studies, I took it with a pinch of salt. The issue on github still has people complaining about performance. – Munim Jul 11 '13 at 11:32
  • 2
    Can you give more (restify) sample code? – Adrian Heine Jul 28 '13 at 09:23
  • @AdrianLang you mean the restify rest app code? it has nothing but a post route which increments a global counter with the new req.s that come in. same replica made in express. – Mithun Satheesh Jul 28 '13 at 10:51
  • And how does the server behave when it stops working? – Adrian Heine Jul 28 '13 at 10:55
  • @AdrianLang: updated my question. sorry for the delay. i was away from my system. – Mithun Satheesh Jul 29 '13 at 11:40
  • Is it just me and my unfamiliarity with restify, or does that `SendMsg` function look like it has suspiciously few callbacks? Are all those POST request operations really supposed to be synchronous calls? – hyde Oct 21 '13 at 20:55
  • @hyde : if I made 1000 http req.s one after the other within a second, does that matter? I dint get your point. – Mithun Satheesh Oct 22 '13 at 03:50

6 Answers6

51

Corrigendum: this information is now wrong, keep scrolling!

there was an issue with the script causing the Restify test to be conducted on an unintended route. This caused the connection to be kept alive causing improved performance due to reduced overhead.


This is 2015 and I think the situation has changed a lot since. Raygun.io has posted a recent benchmark comparing hapi, express and restify.

It says:

We also identified that Restify keeps connections alive which removes the overhead of creating a connection each time when getting called from the same client. To be fair, we have also tested Restify with the configuration flag of closing the connection. You’ll see a substantial decrease in throughput in that scenario for obvious reasons.

Benchmark image from Raygun.io

Looks like Restify is a winner here for easier service deployments. Especially if you’re building a service that receives lots of requests from the same clients and want to move quickly. You of course get a lot more bang for buck than naked Node since you have features like DTrace support baked in.

Stéphane Bruckert
  • 21,706
  • 14
  • 92
  • 130
Masum
  • 1,678
  • 15
  • 19
  • 1
    the blog post you are mentioning is useful, if the author discloses more detail on testing process he followed. See the comments beneath the post! – Mithun Satheesh Mar 17 '15 at 14:14
  • 1
    Yes, that's true, as benchmarking is hard to do correctly, it would be great if the author posted the process and codes. So I took this as a grain of salt and wanted to share with the community. – Masum Mar 17 '15 at 14:22
  • According to the Restify docs, it also supports DTrace. http://mcavage.me/node-restify/#dtrace – Jeff Fairley Apr 14 '15 at 17:34
  • 1
    Also see the [2016 Raygun.io performance test](https://raygun.com/blog/2016/06/node-performance/) – Shane Holloway Nov 17 '16 at 18:34
  • 3
    Please note the Addendum in the same article mentioned before getting into conclusions. – Vignesh T.V. Jan 08 '17 at 10:32
  • This is not up to date anymore. There's 2016 test linked by @ShaneHolloway and Restify performed worse (just get to the end of the article, the author made a mistake during the tests which caused Restify to appear faster). Here are ctual results for 2016 https://raygun.com/blog/wp-content/uploads/2016/06/blogpostamend.png – Maciej Krawczyk Jul 13 '17 at 05:50
28

This is 2017 and the latest performance test by Raygun.io comparing hapi, express, restify and Koa.

It shows that Koa is faster than other frameworks, but as this question is about express and restify, Express is faster than restify.

And it is written in the post

This shows that indeed Restify is slower than reported in my initial test.

enter image description here

OmG
  • 18,337
  • 10
  • 57
  • 90
Puneet Singh
  • 3,477
  • 1
  • 26
  • 39
11

According to the Node Knockout description:

restify is a node.js module purpose built to create REST web services in Node. restify makes lots of the hard problems of building such a service, like versioning, error handling and content-negotiation easier. It also provides built in DTrace probes that you get for free to quickly find out where your application’s performance problems lie. Lastly, it provides a robust client API that handles retry/backoff for you on failed connections, along with some other niceties.

Performance issues and bugs can probably be fixed. Maybe that description will be adequate motivation.

Eric Elliott
  • 4,711
  • 1
  • 25
  • 33
5

I ran into a similar problem benchmarking multiple frameworks on OS X via ab. Some of the stacks died, consistently, after around the 1000th request.

I bumped the limit significantly, and the problem disappeared.

You can you check your maxfiles is at with ulimit, (or launchctl limit < OS X only) and see what the maximum is.

Hope that helps.

  • Hmm.. sounds like it might be similar to the connect.bodyParser() problem, where every connection opens temporary files on the local filesystem? – Eric Elliott Oct 21 '13 at 20:19
  • OSs usually have configurable limits on the number of file descriptors a process, thread and/or the OS can handle simultaneously. For Linux: http://stackoverflow.com/questions/760819/is-there-a-limit-on-number-of-tcp-ip-connections-between-machines-on-linux For MacOS X: http://stackoverflow.com/questions/7578594/how-to-increase-limits-on-sockets-on-osx-for-load-testing – AndreasPizsa Mar 23 '14 at 16:58
5

In 2021, benchmarking done by Fastify (https://www.fastify.io/benchmarks/) indicates that Restify is now slightly faster than Express.

The code used to run the benchmark can be found here https://github.com/fastify/benchmarks/.

a brief summary on how fastify overhead performed against the some other well known Node.js web frameworks

Nick Sinai
  • 135
  • 2
  • 7
3

i was confused with express or restify or perfectAPI. even tried developing a module in all of them. the main requirement was to make a RESTapi. but finally ended up with express, tested my self with the request per second made on all the framework, the express gave better result than others. Though in some cases restify outshines express but express seams to win the race. I thumbs up for express. And yes i also came across locomotive js, some MVC framework build on top of express. If anyone looking for complete MVC app using express and jade, go for locomotive.

kushvarma
  • 369
  • 2
  • 6