6

I've built multiple socket server apps in Node.js for a multi-user artificial intelligence app. We're looking at 1K to 10K active socket connections per box. However even when idle and with 0 active connections, some of my servers consume 50-100 MB of memory when running on Unix. I'm sure with a sensible platform like C# or C++, this should be close to 0 MB. So we are considering a port to a "better" platform. Now let my clarify my use case:

  • This is not a "web server". No files are served.
  • We do lots of CPU intensive data processing and certain portions have already been ported to C++ and pulled into node via native modules.
  • We don't need to access much I/O (in most cases a few files are accessed, in some cases none, we don't use an RDBMS either)

We went with node because it was Unix friendly (unlike .NET) and seemed easy to use. But with its current memory consumption we need to evaluate other options. Many have compared Node.js with ASP.NET but I need to build a socket server in C# or C++.

I have significant experience with .NET and C++. There are libs like SuperSocket (used by Redgate and Telerik) that handle all of the low-level stuff in .NET. I will have to find a similar socket framework for C++.

So putting this all together, what are the advantages of using .NET or C++ over Node.js? And considering my servers are highly CPU-bound (not I/O bound) would the benefits of using .NET/C++ be significant or should I stick with Node.js? Any other comments regarding porting a Node.js app to C# or C++?

Bounty: I need advice and a recommended socket server library/implementation/example app in C# and/or C++. Must be open source. I need it to be high-performance, async and bug-free. Must support binary data transfer. Must run on Windows. Unix is a bonus.

Robin Rodricks
  • 110,798
  • 141
  • 398
  • 607
  • However even when idle and with 0 active connections, some of my servers consume 50-100 MB of memory -- why does it matter how much memory it's using when idle? – xaxxon May 02 '16 at 22:27
  • Questions asking us to recommend or find a book, tool, software library, tutorial or other off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it. – xaxxon May 02 '16 at 22:28
  • Although I understand your general concern of doing something right, 50-100M seems almost nothing to me. Todays server have hundreds of GB or RAM: https://nickcraver.com/blog/2016/03/29/stack-overflow-the-hardware-2016-edition/ otherwise you could have a look at zeroMQ and nanomsg, also NetMQ and AsyncIO (which leverage Windows IO completion ports, if you're running on Windows): http://somdoron.com/2014/11/netmq-iocp/ at least to understand how they work before you roll your own thing – Simon Mourier May 03 '16 at 06:38
  • possibly helpful if you want to make your own socket server-client async. it has the example of working program too.. Not designed for highly dense application though: http://stackoverflow.com/questions/34586733/sending-a-value-from-server-to-client-with-sockets/34669446#34669446 and http://stackoverflow.com/questions/36600012/how-to-send-file-through-socket-in-c-sharp/36637326#36637326 – Ian May 03 '16 at 07:31
  • @xaxxon - Because we need to run as many server apps on one box as possible, most of which will be idle, this is because we have different apps for different services, and need as many services on one box as possible. – Robin Rodricks May 03 '16 at 14:58
  • the only thing that matters is how it scales, not how much it uses when idle. – xaxxon May 03 '16 at 17:23
  • Did you consider tornado (it is an async python web server). Very light and friendly. Running C++ code from it will be easy – Avia May 03 '16 at 22:10
  • @Avia - I've already used one low-performance platform which is Node.JS and I'm already using C++ modules. I have bad performance and bad memory usage. I don't want to try another newfangled thing. I want something stable and strongly typed, and high-performance. Pure C++ or C#+C++, not python, not node, not JS, not PHP. – Robin Rodricks May 04 '16 at 08:33
  • Checkout NetMQ for Windows and C# it is fast, mature and using IOCP. If you are using C++ I suggest linux/unix and zeromq. – somdoron May 05 '16 at 13:13
  • @somdoron - Your answer sounds interesting. Can you please elaborate and add your comments as an answer? – Robin Rodricks May 06 '16 at 09:23

3 Answers3

9

We're looking at 1K to 10K active socket connections per box

the bottleneck here is not the programing language or the technology, it's the hardware and OS support. the thing that limits the amount of concurrent sockets count is basically the machine you're running on. yet, from my experience, the determinisitic object lifetime of C++ can help dramatically for supporting large number of concurrent OS resources.

This is not a "web server". No files are served.

I have done some Node.js in my profesional work, I have done some C# but mostly C++. even with node.js as a web server, most of the client and server code didn't had many much in common besides the language itself. the web server dealt with buisness logic mostly, while the client dealt with fetching and presenting the data interactivly. So, I think the main advantage of node.js as a web server is that it gives purist-JS developers the ability to write server side without using languages/technology they are not familliar with.

We do lots of CPU intensive data processing and certain portions have already been ported to C++ and pulled into node via native modules.

yep. using strongly typed language can do wonders here. no redunadand runtime-parsing.

We don't need to access much I/O (in most cases a few files are accessed, in some cases none, we don't use an RDBMS either)

Well, I feel there's a myth in the air that node.js somehow handles IO better than other technologies. this is simply wrong. the main feature of Node.js is the fact that by default, the IO is asynchronous. but Node.js didn't invent any wheel. you have asynchronous IO in Java (aka Java.NIO), C# (async/await) and C++ (as native stuff like epoll/IOCompletionPort, or some higher stuff like Boost.ASIO/ CPP-rest, Proxygen etc.)

We went with node because it was Unix friendly (unlike .NET)

.Net Core is a relativly new technology where .Net can run on Unix-based systems (like linux)

I will have to find a similar socket framework for C++.

Boost.ASIO, or write something yourself, it's really not that hard..

So putting this all together, what are the advantages of using .NET or C++ over Node.js?

better CPU usage: because C++ and C# are strongly typed languages, and C++ is a statically compiled language, there are huge oppretunities for the compiler to optimize CPU extensive jobs.

lower memory footprint: usually because strongly typed languages have smaller objects without the overhead of keeping a lot of meta-data behind the scences. with C++, having stack allocation and scoped object life-time usually the memory footprint is low. again, it depends on the quality of the code in any language.

no callback hell: C# has tasks and async await. C++ has futures/promises and some compilers (aka VC++) do supports await as well. the asynchronous code simply becomes pure fun to write as oppossed to callbacks. yes, I do aware of JS promises and the new async/await stuff, but they are relativly new compared to .Net implementation.

Compiler checks : since C# and C++ have to be compiled, a lot of silly bugs are caught in compile time. no "undefiend is not a function" or "cannot read property of undefined".

other than that it's pretty much a matter of choice.

David Haim
  • 25,446
  • 3
  • 44
  • 78
  • Beautiful answer! I am only keeping the question open to get more info... else I would have accepted your answer immediately. – Robin Rodricks Apr 18 '16 at 11:19
  • 1
    thanks you. I already tell you that this question is opinion based so other sites, like stackexchange.com may be more suitbale than stackoverlfow – David Haim Apr 18 '16 at 11:36
  • Pretty solid answer from David Haim. As for a socket library, I would suggest to have a look at the ZeroMQ library (http://zeromq.org) to see if it fits your architecture, both for C++ and C#. – Ton Plooij Apr 30 '16 at 20:53
  • @DavidHaim - Can you PLEASE provide some socket server example app code for C++, which is async and reliable? I will award you the bounty but I need some starter code to work with. – Robin Rodricks May 04 '16 at 08:38
  • @HarshGupta I will soon. – David Haim May 05 '16 at 08:06
  • @DavidHaim - Can you PLEASE provide some socket server example app code for C++, which is async and reliable? – Robin Rodricks Jul 26 '16 at 07:24
  • 1
    @HarshGupta here is an example of a Boost.ASIO echo TCP server : http://www.boost.org/doc/libs/1_55_0/doc/html/boost_asio/example/cpp11/echo/async_tcp_echo_server.cpp – David Haim Jul 27 '16 at 12:28
  • 1
    @HarshGupta and a C# one : https://msdn.microsoft.com/en-us/library/fx6588te(v=vs.110).aspx – David Haim Jul 27 '16 at 12:29
  • @DavidHaim - Thank you very much. I'll check those out. – Robin Rodricks Jul 29 '16 at 15:59
2

NetMQ is native C# port of zeromq.

Zeromq is lightweight messaging library, the zeromq guide is a great if you want to learn about messaging, it also come as a book. It applicable both to zeromq and NetMQ.

If you are using windows and need to handle a lot of connection I don't recommend zeromq as it not using IOCP.

NetMQ is using IOCP on Windows and works both on windows and linux.

Disclosure - I'm author of NetMQ and maintainer on the zeromq (libzmq) project.

[1] https://github.com/zeromq/netmq

[2] http://netmq.readthedocs.io/en/latest/

[3] http://zguide.zeromq.org/page:all

[4] http://www.amazon.com/ZeroMQ-Messaging-Applications-Pieter-Hintjens/dp/1449334067/ref=sr_1_1?ie=UTF8&qid=1462550951&sr=8-1&keywords=zeromq

somdoron
  • 4,653
  • 2
  • 16
  • 24
  • Can you send a link for a sample socket server app built with NetMQ? Is NetMQ stable? Any apps using it? – Robin Rodricks May 07 '16 at 06:55
  • NetMQ is stable, mature, and active with a lot of users (around 70K downloads on nuget). Also some other open source project are built on NetMQ, search nuget. Regarding example, take a look at the following repository, it just an example on how to write NetMQ components. https://github.com/somdoron/NetMQ.High – somdoron May 07 '16 at 06:59
1

We do lots of CPU intensive data processing

Node.js may have been the wrong choice from the start and it would probably never match performances of a C++ server. However, it can be pretty close, if you are doing things right. In addition, writing good C++ and a complete rewrite of a system is difficult and time consuming. So, I want to give some reasons for you to stick to Node.js or at least, completely exhaust all your options before you move.

my servers consume 50-100 MB

Are you using Node.js v0.12? With Node.js v4.2 LTS, idle Node.js server should use around 20 MB of memory. (It would probably never be near 0 MB because of V8) Have you checked for memory leaks?

1K to 10K active socket connections per box

This should be easily achievable. If you are using the most popular socket.io library, here's some relevant benchmarks.

on a 3.3 GHz Xeon X5470 using one core, the max messages-sent-per-second rate is around 9,000–10,000 depending on the concurrency level.

from: http://drewww.github.io/socket.io-benchmarking/ (Since, all these connections are kept alive concurrently, CPU usage matters more)

If you are already using that and having issues, try replacing socket.io with SocketCluster which is faster and more scalable. Replacing this should be easier than a complete rewrite. Here's some benchmarks:

8-core Amazon EC2 m3.2xlarge instance running Linux

at 42K, the CPU use of the busiest worker dropped to around 45%

http://socketcluster.io/#!/performance

Finally, to prove that Node.js can nearly reach C++ performance. Have a look at this:

servers use 12G memory

It supports 1,200,000 active websocket connections

https://github.com/smallnest/C1000K-Servers

My point is you have average performance goals that you should be able to reach with Node.js with little effort. Try to benchmark (https://github.com/machinezone/tcpkali) and find the issue rather than do a complete rewrite.

Community
  • 1
  • 1
Rahat Mahbub
  • 2,967
  • 17
  • 29
  • I'm using Node 0.12, and we are using raw sockets (ie without any library), just the stuff node provides. Won't that be better?. We are not using websockets. We have a custom .NET socket client app, not HTML. Memory benchmarking is nearly impossible as all the memory debugging tools (WebStorm, Chrome Remote debugging, etc) show only Array and Object and a lot of other weird stuff. Types are not available so we cant really see which kind of object is consuming memory. – Robin Rodricks Apr 29 '16 at 08:08
  • One of our servers is a simple ~100 LOC thing which only does some brief socket communication, and simple file I/O, and it takes anywhere from 30 MB to 60 MB of memory when idle. This is intolerable. Would upgrading the node version help this? – Robin Rodricks Apr 29 '16 at 08:12
  • Yes. Node v0.12 had an issue. Please upgrade to latest LTS. Node has great profiling tools now. Google search should tell you. If the server code is only 100 LOC and from your question and comments, I feel that you strongly dislike node/JavaScript but was forced to use it. So, please go ahead and use something that you enjoy and have experience in. – Rahat Mahbub Apr 29 '16 at 08:40
  • I'm also confused by your first comment. If you are not using web sockets, do you mean TCP sockets by raw sockets? – Rahat Mahbub Apr 29 '16 at 08:47
  • Yes TCP sockets. No dislike here. I loved node and chose it myself (.NET/C++ was not an option at the time). Please specify which profiling tools can be used. I used WebStorm etc and all I get is untyped objects. The main size seems to be taken by the Node.js runtime (code cache or something, don't remember the exact name that showed up) – Robin Rodricks Apr 29 '16 at 16:47
  • Please reply if you want the bounty. Please specify which profiling tools can be used. I used WebStorm etc and all I get is untyped objects. Like this : http://blog.jetbrains.com/webstorm/2015/07/node-js-profiling-in-webstorm-part-2-memory-profiling/ – Robin Rodricks May 04 '16 at 08:36