8

We have an application server developed with Delphi 2010 and Indy 10. This server receives more than 50 requests per second and it works well. But in some cases, it seems to me that Indy is very obscure. Their components are good, but sometimes I found myself digging into the source code only to understand a simple thing. Indy lacks on good documentation and good support.
The last thing that i came across was a big problem for me: I must detect when a client disconnects non gracefully (When the the client crashes or shutdown, for instance. Not telling the server that it will disconnect) and indy was not able to do that. If I want that, I will have to develop a algorithm like heartbeat, pooling or TCP keep-alive. I do not want to spend more time doing a, at least I think, component job. After a few study, I found out that this is not Indy's fault, but this is an issue of all blocking sockets components.

Now I am really thinking of changing the core of the Server to another good suite. I must admit I am tending to use a non-blocking socket. Based on that, I have some questions:

  • What do a benefit from changing from blocking to non-blocking sockets?
  • Will I be able to detect client disconnects (non gracefully)?
  • What component suite has the best product? By best product I mean: fast, good support, good tools and easy to implement.

I know this must be a subjective question, but I really want to hear that from you. My first question is the one I care most. I do not care if I have to pay 100, 500, 1000, 10000 dollars, but I want a complete solution. For now, I am thinking about Ip*works .

EDIT

I think some guys are not understand what I want. I don't want to create my own socket. I have been working with sockets for a long time and I am getting tired of it. Really.

And non-blocking sockets CAN detect client disconnects. That is a fact and it has good documentation all over the internet. A non-blocking socket checks the socket state for new incoming data all the time, and it makes possible to detect that the socket is not valid. This is not a heartbeat algorithm. A heartbeat algorithm is used on client side and it sends periodically packets (aka keep-alive) to the server to tells it is still alive.

EDIT

I am not make myself clear. Maybe because English is not my main language. I am not saying that it is possible to detect a dropped connection without trying to send or receiving data from a socket. What I am saying is that every non-blocking socket is able to do that because they constantly tries to read from the socket for new incoming data. Why is that so hard to understand? If you guys download and run ip*works demos, in special, the echoserver and echoclient ones (both use TCP) you can test by yourselves. I already tested it, and it works like I expected to do. Even if you use the old TCPSocketServer and TCPSocketClient in a non-blocking mode you will see what I meant.

Rafael Colucci
  • 6,018
  • 4
  • 52
  • 121
  • What do mean exactly by disconnects non-gracefully? – Duck May 06 '11 at 03:13
  • When the the client crashes or shutdown, for instance. Not telling the server that it will disconnect. – Rafael Colucci May 06 '11 at 03:16
  • 2
    Non-blocking sockets CANNOT detect dropped connections any more than blocking sockets can - see my answer below. – Misha May 06 '11 at 14:42
  • 1
    PS Indy does a really good job disconnecting the server if the underlying connection really is dropped due to the client closing down normally. I have some really robust TCP server stuff that has been running for years without a problem with "dropped" connections. And lastly, I think there are far more advantages using blocking sockets on the server - it makes for a cleaner, more scalable architecture so my advice to you is NOT to change. – Misha May 06 '11 at 14:53
  • 1
    You can do the same thing in Indy, for both client and server. Just use IoHandler.CheckForDataOnSource and then read IoHandler.InputBuffer.Size bytes from the socket. I have this running on a 10ms loop for every connection. So what is that you cannot do with Indy? – Misha May 06 '11 at 15:10
  • @Misha. I will try that, thanks. But that is exactly the indys problem. It is hard to find information as the one you just gave me. If i have a good component with good support maybe I would not have to go to stackoverflow to know about it. – Rafael Colucci May 06 '11 at 15:14
  • @misha do you call IoHandler.CheckForDataOnSource in a loop in another thread? Do you do that to check if the client is still connected? – Rafael Colucci May 06 '11 at 15:16
  • I call IoHandler.CheckForDataOnSource to see if there is any data available, and then use IoHandler.InputBuffer.ExtractToBytes to get any available data. All in the same thread as I use for writing. If you download my code from http://www.csinnovations.com/framework_delphi.htm and have a look at the AppTcpServerUnt.pas file you will be able to see exactly how this is all done (or I could send you that unit for you to have a look at). – Misha May 06 '11 at 15:23
  • @Rafael, some confusion here might be because I am using Indy in a "non-blocking way". I read and write with the same thread on a tight timer and this works well for 100s of concurrent connections and my threading framework. Perhaps this is why I had trouble seeing what you could not do. – Misha May 06 '11 at 15:34
  • @Rafael - pls-let's not confuse blocking and non-blocking sockets in general with particular implementations thereof. As @Misha and I explained, TCP/IP itself is the same regardless of whether it's a blocking or non blocking socket; a special mechanism is needed to detect a dead client. Particular implementations of blocking and non-blocking sockets can each handle this situation, in the manner most appropriate for that implementation and blocking model. Your question was general - you are considering many different implementations and wanted to know about blocking and non-blocking in general. – Vector May 06 '11 at 19:20
  • @mikey I agree with you. But what i said is that most of non-blocking sockets implementations have already an way of detecting dead clients because they are trying to read and manipulate the socket all the time. – Rafael Colucci May 06 '11 at 19:27
  • @Rafael - granted that you are correct that NBSckt implementations make it easier to deal with dead clients (I rarely use NBS-haven't in years-so I can't really say) but it's not really very difficult with BSckts either - I think others here have already explained some of that. And on a server like you described, IMO you'll have a lot more work getting NBSckts working well than you will impmelementing something with your current Indy implementation and very seriously doubt that you'll have any improvement in performance using NBSckts - I tend to think just the opposite; just my $0.02... – Vector May 06 '11 at 21:10
  • 1
    @Rafael, regardless of the other points, you CANNOT detect dead clients by READING from the socket - to do this you must WRITE to it. Reading is a passive operation and cannot detected dropped connections. – Misha May 06 '11 at 23:17

6 Answers6

15

"What do a benefit from changing from blocking to non-blocking sockets?
Will I be able to detect client disconnects (non gracefully)?"

Just my two cents to get the ball rolling on this question - I'm not a socket EXPERT, but I do have a good deal of experience with them. If I'm mistaken, I'm sure someone will correct me... :-)

I assume that since you're running a server using blocking sockets with 50 connections per second, you have a threading mechanism in place to handle client requests. If so, you don't really stand to gain anything from non-blocking sockets. On the contrary - you will have to change your server logic to be event driven- based on events fired in your main thread from the non-blocking sockets, or use constant polling to know what your sockets are up to.

Non-blocking sockets can't detect clients disconnecting without notification any more than blocking sockets can - they don't have telepathic powers... The nature of the TCP/IP 'conversation' between client and server is the same - blocking and non-blocking is only with respect to your application's interaction with the socket connection conducting the 'conversation'.

If you need to purge dead connections, you need to implement a heartbeat or timeout mechanism on your socket (I've never seen a modern socket implementation that didn't support timeouts).

Vector
  • 10,879
  • 12
  • 61
  • 101
  • No. I am not using threads at all. Indy main read event is called from a thread, which means I dont have to care about it. I also dont care if I have to change the server logic in order to use non blocking sockets, as long as i benefit from doing it. I am pretty sure non-blocking sockets can detect client disconnections because they can check for the socket state from time to time sending small packages to them. – Rafael Colucci May 06 '11 at 04:26
  • -1, check this out: https://forums.embarcadero.com/thread.jspa?threadID=49732. It proves that non-blocking sockets can detect client disconnections. – Rafael Colucci May 06 '11 at 04:34
  • 3
    I think your -1 is unwarranted - "for the socket state from time to time sending small packages to them" - that's called a heartbeat and a blocking socket can do the same. I suggest you read that thread again. I also began with a disclaimer. Thanks. – Vector May 06 '11 at 05:41
  • No threads on the server side? This means all requests are handled sequentially, and the client(s) have to wait until a server socket is available. Or did I misunderstand? – mjn May 06 '11 at 06:27
  • 1
    @mjn. Indy has its own thread pool. Each request is handled in its own thread context. What he meant is that he does not have his own threads implemented but uses Indy thread pool instead which is build into Indy. Yes the application is still multithreaded :) – Runner May 06 '11 at 08:39
  • @Mikey, no, this is not heartbeat. The non-blocking socket can check the socket state even without sending a packet to the client. It can query the socket state. Hes able to do that because they constantly checks the socket for new incoming data. – Rafael Colucci May 06 '11 at 13:08
  • @runner, yeah .. thats what i meant. Thanks. – Rafael Colucci May 06 '11 at 15:02
5

What do a benefit from changing from blocking to non-blocking sockets?

Increased speed, availability, and throughput (from my experience). I had an IndySockets client that was getting about 15 requests per second and when I went directly to asynchronous sockets the throughput increased to about 90 requests per second (on the same machine). In a separate benchmark test on a server at a data-center with a 30 Mbit connection I was able to get more than 300 requests per second.

Will I be able to detect client disconnects (non gracefully)?

That's one thing I haven't had to try yet, since all of my code has been on the client side.

What component suite has the best product? By best product I mean: fast, good support, good tools and easy to implement.

You can build your own socket client in a couple of days and it can be very robust and fast... much faster than most of the stuff I've seen "off the shelf". Feel free to take a look at my asynchronous socket client: http://codesprout.blogspot.com/2011/04/asynchronous-http-client.html

Update:
(Per Mikey's comments)

I'm asking you for a generic, technical explanation of how NBS increase throughput as opposed to a properly designed BS server.

Let's take a high load server as an example: say your server is supposed to handle 1000 connections at any given time, with blocking sockets you would have to create 1000 threads and even if they're mostly idle, the CPU will still spend a lot of time context switching. As the number of clients increases you will have to increase the number of threads in order to keep up and the CPU will inevitably increase the context switching. For every connection you establish with a blocking socket, you will incur the overhead of spawning of a new thread and you eventually you will incur the overhead of cleaning up after the thread. Of course, the first thing that comes to mind is: why not use the ThreadPool, you can reuse the threads and reduce the overhead of creating/cleaning-up of threads.

Here is how this is handled on Windows (hence the .NET connection): sure you could, but the first thing you'll notice with the .NET ThreadPool is that it has two types of threads and it's not a coincidence: user threads and I/O completion port threads. Asynchronous sockets use the IO completion ports which "allows a single thread to perform simultaneous I/O operations on different handles, or even simultaneous read and write operations on the same handle."(1) The I/O completion port threads are specifically designed to handle I/O in a much more efficient way than you would ever be able to achieve if you used the user threads in ThreadPool, unless you wrote your own kernel-mode driver.

"The com­ple­tion port uses some spe­cial voodoo to make sure only a spe­cif­ic num­ber of threads can run at once — if one thread blocks in ker­nel-​mode, it will au­to­mat­i­cal­ly start up an­oth­er one."(2)

There are other advantages also: "in addition to the nonblocking advantage of the overlapped socket I/O, the other advantage is better performance because you save a buffer copy between the TCP stack buffer and the user buffer for each I/O call." (3)

Kiril
  • 39,672
  • 31
  • 167
  • 226
  • Thanks for sharing, but your answer is based on client side point of view. I am sure I can build my own non blocking socket, but I have no time for that and the company I work for does not care if I want to buy a suite, as long as I justify why I need such thing. – Rafael Colucci May 06 '11 at 04:30
  • @Rafael, I understand your situation, but I think that you'll find that most of the suites out there are all based on the .NET sockets and the amount of code you'll have to write in order to use THEIR wrapper around .NET sockets is about the same as just using the .NET sockets. If my answers is not 100% what you're looking for, it may still be useful down the line if you encounter the same problems that I did with regards to the 3rd party solutions. – Kiril May 06 '11 at 11:49
  • @lirik, I appreciate your answer, i really do. It helped me. But i must disagree. There are good delphi suites over there. ipworks is one of then. I really dont want to waste my time doing a thing that somebody else may did already. – Rafael Colucci May 06 '11 at 13:07
  • @Rafael no problem, I don't generally encourage the reinventing the wheel either, but it all depends on how much performance you need to squeeze out of a 3rd party solution. Good luck! – Kiril May 06 '11 at 13:33
  • @Lirik - How will non blocking increase speed? My understanding is that **in general** blocking sockets using threads are more appropriate for servers and non blocking are better suited to front-end client apps. – Vector May 06 '11 at 19:18
  • @Mikey the asynchronous methods in .NET are intended for servers which have excessive loads: http://stackoverflow.com/questions/5202608/is-there-any-performance-difference-between-begin-and-async-for-sockets-in-net and http://stackoverflow.com/questions/5764921/whats-the-difference-between-beginconnect-and-connectasync – Kiril May 07 '11 at 13:38
  • @lirik - DK about .NET and asynchronous methods, DK how those links relate to the discussion, particularly since this is a Delphi question. I'm asking you for a generic, technical explanation of how NBS increase throughput as opposed to a **properly designed** BS server. You cited no particulars about your architecture, just that you used Indy and switched to non-blocking and perceived a difference. What is better/different about nbs that you say caused such an increase in throughput. How do we know it's because of nbs - maybe it's due to your architecture or use case? – Vector May 07 '11 at 18:05
  • @lirik - thanks for the informative update. But what you've shown us is that the key is I/O completion ports, not NBSckts, and that there is a particular type of thread pool in .NET that leverages I/O completion ports. I Googled around a bit and found this page which discusses winSock I/O models: (http://www.madwizard.org/programming/tutorials/netcpp/5) - not light reading, but worth it. – Vector May 08 '11 at 01:29
  • @Mikey In general you would still have to address the issues tho: your blocking sockets require 1 thread per connection, so you can only scale up so much (i.e. as much as your cores allow). – Kiril May 08 '11 at 02:36
  • @lirik - Granted - but nbsckts also have overhead because of their message processing. Regardless, I personally have used blocking sockets and threading for many years and I favor the simplicity of their programming model and scalability. And processing power today is cheap: My boss just bought himself a 16 core 64bit Win 7 workstation with 16 GB RAM for $1800. – Vector May 08 '11 at 03:21
  • @Mikey, yes the processing power is cheaper, but you still want to get the most out of it. – Kiril May 08 '11 at 06:19
  • 90 to 300 requests per second is not much. With Indy 10.6, my client libraries easily reach 40.000 messages roundtrips - send and receive - per second (continuously), with the server running on the same VM. – mjn Oct 03 '14 at 13:00
3

I am using Indy and Synapse TCP libraries with good results for some years now, and did not find any showstoppers in them. I use the libraries in threads - client and server side, stability and performance was not a problem. (Six thousand request and response messages per second and more with the server running on the same system are typical.)

Blocking sockets are very useful if the protocol is more advanced than a simple 'send a string / receive a string'. Non-blocking sockets cause a higher coupling of message protocol handlers with the socket read / write logic, so I quickly moved away from non-blocking code.

No library can overcome the limitations of the TCP/IP protocol regarding detection of connection loss. Only trying to read or send data can tell wether the connection is still present.

mjn
  • 36,362
  • 28
  • 176
  • 378
  • As I said, non-blocking sockets can do that. They check the socket state from time to time. – Rafael Colucci May 06 '11 at 14:16
  • 1
    @Rafael, non-blocking sockets are no different to blocking sockets with regards to detecting blocked connections. The TCP protocol was specifically designed so that connections are NOT dropped on unplugged network cables and router crashes. This is so that the original connection can be reused when the cable is plugged back in or the router comes back up. Please, please read the link I have put in my answer. – Misha May 06 '11 at 14:49
3

In Windows, there is a third option which is overlapped I/O. Non-blocking sockets are essential a model using Windows messages developed to avoid single-threaded GUI apps to become "blocked" while waiting for data. A modern application IMHO would be better designed using threads and overlapped I/O.

See for example http://support.microsoft.com/kb/181611

3

Aahhrrgghh - the myth of being able to always detect "dropped" connections. If you pull the power on a machine with a client connection then the server cannot tell, without sending data, that the connection is "dead". The is through the design of the TCP protocol. Don't take my word for it - read this article (Detection of Half-Open (Dropped) TCP/IP Socket Connections).

Misha
  • 1,816
  • 1
  • 13
  • 16
  • I am not saying that it is possible to detect dropped connections without sending or checking the socket. I am saying that every non-blocking socket does that by constantly reading the socket from new incoming data. Why is that so hard to understand? – Rafael Colucci May 06 '11 at 14:54
  • 1
    @Rafael, firstly you can only detect dropped connections by sending data. Secondly, the same functionality can be achieved using Indy. I have had this code running for years, and you can find it on the web. – Misha May 06 '11 at 15:03
  • 1
    And can someone tell me why I was downvoted for being correct! – Misha May 06 '11 at 15:05
  • There is still confusion around regarding the difference between blocking and non-blocking. I have read the introduction to Indy so I know that blocking is NOT evil :) - http://www.swissdelphicenter.ch/en/showarticle.php?id=4 - and there is no reason why non-blocking sockets are 'smarter' in detecting socking state – mjn May 06 '11 at 15:18
2

This article explains the main differences between blocking and non-blocking:

Introduction to Indy, by Chad Z. Hower

Pros of Blocking

  • Easy to program - Blocking is very easy to program. All user code can exist in one place, and in a sequential order.
  • Easy to port to Unix - Since Unix uses blocking sockets, portable code can be written easily. Indy uses this fact to achieve its single source solution.
  • Work well in threads - Since blocking sockets are sequential they are inherently encapsulated and therefore very easily used in threads.

Cons of Blocking

  • User Interface "Freeze" with clients - Blocking socket calls do not return until they have accomplished their task. When such calls are made in the main thread of an application, the application cannot process the user interface messages. This causes the User Interface to "freeze" because the update, repaint and other messages cannot be processed until the blocking socket calls return control to the applications message processing loop.

He also wrote:

Blocking is NOT Evil

Blocking sockets have been repeatedly attacked with out warrant. Contrary to popular belief, blocking sockets are not evil.

It is not is an issue of all blocking sockets components that they are unable to detect a client disconnect. There is no technical advantage on the side of non-blocking components in this area.

mjn
  • 36,362
  • 28
  • 176
  • 378