22

I've not been doing bare metal TCP/IP for about 18 months, so I wonder what the current state of the art is.

I'm looking for both positive and negative aspects, with development of both server and client software.
I will be doing a project that needs a rock-solid TCP/IP layer, so for me that is an important aspect :)

For this to become a community wiki, I'm looking for broader answers than just 'rock solid'. So for instance information about the feature-width is also appreciated.

I'll be updating the question with relevant aspects found in the answers in order to get a wiki entry that has a balanced overview of those libraries.

For example, see my answer below with my past experience with Indy I'm ambivalent on the exception handling and anti-freeze in Indy, though I got used to it, it still felt somewhat unnatural.

Right now I develop in both Delphi 2007 (non Unicode) and XE (Unicode), so the libraries I'm considering should support at least those two Delphi versions.

Toon Krijthe
  • 52,876
  • 38
  • 145
  • 202
Jeroen Wiert Pluimers
  • 23,965
  • 9
  • 74
  • 154
  • 3
    +1, don't see why the -1, looks like a valid and well formated question? – Johan Apr 18 '11 at 21:32
  • @Johan: It shouldn't matter for community wikis: http://meta.stackexchange.com/questions/11740/what-are-community-wiki-posts – Jeroen Wiert Pluimers Apr 18 '11 at 21:37
  • 1
    One comment - raw TCP has barely changed in the last 20 years. If implementing over raw TCP then features are pretty well irrelevant. If, however, you are implementing a higher-level protocol, then features may be the determining factor. – Misha Apr 19 '11 at 00:15
  • 1
    IOCP support (since NT 3.5) is interesting for large scale apps and is partially available : http://stackoverflow.com/questions/2302267/is-there-a-i-o-completion-port-based-component-for-delphi – mjn Apr 19 '11 at 14:20
  • Unfortunately "partially" may as well be "not at all" in the case of Delphi components. Would really like to see a robust implementation of IOCP in Delphi, but I will not be holding my breath ;-) – Misha Apr 19 '11 at 14:32

7 Answers7

9

Edit: Summary of my past experience with Indy, and the comments (thanks Eugene, Marjan) (please update with the current Indy state of the art):

Pro:

  • ships with Delphi
  • mature
  • development community
  • open source so lots of eyes scrutinizing those sources
  • a truckload of valuable comment documentation in the source code
  • OpenSSL support
  • supports a broad set of Delphi versions (including 2007 and XE)
  • wide choice of protocols

Con:

  • the version shipping with Delphi was not always the most stable one; download from the sources was usually required to get a stable build
  • (in the mean time) lots of duplication of code that now is in Delphi (but Indy requires for compatibility with older Delphi versions)
  • not all TCP/IP components were up-to-date (for instance, back then the POP3 client component did not support some basic POP3 commands)
  • version interoperability was a pain: upgrading from one Indy version to another could be very time consuming
  • I'm ambivalent on the exception handling and anti-freeze in Indy, though I got used to it, it still felt somewhat unnatural.
  • breaking changes are made between build updates; ifdefs required to accommodate those
  • Unclear release status if any at all, no RCs for a long while, getting trunk can make your local copy unstable
Community
  • 1
  • 1
Jeroen Wiert Pluimers
  • 23,965
  • 9
  • 74
  • 154
  • 1
    Add to pros - wide choice of protocols. Add to cons - breaking changes are made between build updates. We have a bunch of ifdef's in our Indy helper classes to accommodate to those breaking changes. – Eugene Mayevski 'Callback Apr 19 '11 at 06:14
  • 3
    Add to cons: Unclear release status if any at all. There haven't been Release Candidates for a long time or even buildnumber "marked" as stable. And always getting the latest source is not an option as it introduces instability in your own product. – Marjan Venema Apr 19 '11 at 07:47
  • @Eugene, Marjan: thanks ,updated. Note that you are encouraged to update the answer as it is a community wiki answer. – Jeroen Wiert Pluimers Apr 19 '11 at 12:19
5

ICS - The Internet Component Suite

ICS - see www.overbyte.be. Open source by François Piette. To me this has always been the number 1 alternative to Indy. It's most interesting selling point: it makes using asynchronous stuff easy, and being async seems to be closer to "bare metal" sockets programming.

I've used it to build a fairly complex VNC proxy where the proxy itself (server) is built with ICS and the clients are a mixture of Indy and ICS. In periods of high demand the proxy handles about 100 simultaneous connections and about 10 simultaneous VNC screen sessions. It eats up an average of 5 mbit/s, handles connections over two distinct Internet connections. I don't think the 100 + 10 is the limit, because the server handles that without any problems and CPU usage is too low to mention.

Pros:

  • Works asynchronously
  • Somewhat easier on beginners because it doesn't need threads
  • Supports a good number of protocols

Cons:

  • Relies on Windows messaging. I'm simply not comfortable with that.
  • The async behavior makes implementing most protocols slightly difficult (because most protocols are in the form of send command / receive response). This shouldn't matter for most people since ICS offers ready-made implementation for the most-used protocols.

All that being said, I haven't used ICS in a very long time, I'm not up-to-date with all the bells and whistles. This is CW, so please edit and expand!

Community
  • 1
  • 1
Cosmin Prund
  • 25,498
  • 2
  • 60
  • 104
2

I have used Indy since 2003 for my own TCP communications framework. It is rock-solid, I have a version used with Delphi 2007 and another with Delphi 2010, if you handle the threadng correctly there is no need to use the anti-freeze stuff, and I have consistent exception handling on the client and the server by implementing my own wrapper around this.

You can dowload it here (http://www.csinnovations.com/framework_delphi.htm) - look for the Tcp units, mainly AppTcpServerUnt and AppTcpClientUnt.

Misha
  • 1,816
  • 1
  • 13
  • 16
  • 2
    And after checking, one of my servers that has been up for the last 3 months has processed 140 million messages and a total of 40 GB of data since startup. – Misha Apr 18 '11 at 23:22
  • I want to try your framework, but need to understand how to a) stop a thread, b) work with UI, c) send message to another thread, d) send a dataset to the client. Can you framework do this? Thanks. – Clóvis Valadares Junior Apr 19 '11 at 01:15
  • Have a look at the development guide, and then have a look at some of the examples in the 4Framework directory. This will give you a bit of background information. Email me at the address onthe web site if/when you need more information. – Misha Apr 19 '11 at 01:38
  • This sounds really useful, but the link is broken now. Could you please update the link to it's new location? – Neville Cook Jun 12 '16 at 22:06
2

I would strongly recommend Clever Internet Suite, it's by far the best designed and written set of communication components. It's not free and so not that well known, but it's well worth investigating.

Pro:

  • well designed and written
  • contains many components and implements various protocols.
  • supports a broad set of Delphi versions (including 2007 and XE)
  • SSL support
  • mature product as the release history indicates

Con:

  • not open source
2

The answer really depends on many factors and your requirements, such as

  1. what layers are needed (TCP, SSL/TLS, application-level protocols)
  2. whether you need a client or a server as well (server is much more complicated task)
  3. whether you count paid options.

In general, not much (positive) happened in 18 months or even in 3 years as most developers look at .NET as primary development platform.

Clever Internet Suite mentioned in other answer and DevArt's SecureBridge gained some new functionality.

Our SecureBlackbox offers support for the most advanced features (besides native SSL/TLS): IPv6, HTTPS Proxy with basic, digest and NTLM authentication (starting with SecureBlackbox 9), International Domain Names (starting with SecureBlackbox 9), DNSSEC, bandwidth control and more.

Application-level protocols supported by SecureBlackbox are HTTP (client and server), WebDAV (client and server), FTP (client and server), SSH and SFTP (client and server), SMTP and POP3 clients, DNS client, AS2 and AS3. All of the protocols (besides SSH and SFTP, of course) have complete support for SSL/TLS.

The list of supported protocols can be found on Packages page. Supported protocol features are listed on Technical Specification page for each package.

Eugene Mayevski 'Callback
  • 45,135
  • 8
  • 71
  • 121
  • From the package pack of SBB, it is unclear if other protocols than security related protocols are being supported (what about HHTP, POP3, TELNET, DNS, just to name a few)? http://www.eldos.com/sbb/packages.php – Jeroen Wiert Pluimers Apr 19 '11 at 05:26
  • @Jeroen HTTP, FTP and SMTP are available, WebDAV and POP3 are available in version 9 (now in public beta), DNS is available together with DNSSEC. Still, merely comparing the number of protocols (either with other product or with one's expectations) is not exactly correct because implementations of the same protocol can differ in supported features significantly. – Eugene Mayevski 'Callback Apr 19 '11 at 05:34
  • I understand your point, and it was in fact one of the major time consuming things in Indy: finding out what parts of the protocols are/aren't supported; is there documentation which protocols and which coverage of those protocols SBB supports? – Jeroen Wiert Pluimers Apr 19 '11 at 05:48
  • @Jeroen that's a good question :). We implement the protocols in full. Unfortunately there's no single place to see the list of all components / features, but there's a list of components grouped into packages (http://www.eldos.com/sbb/packages.php ) and a list of features per package (see Technical Specification section for each package). Also the help file lists all methods and properties of all components, so by browsing through the list you can find the needed function. – Eugene Mayevski 'Callback Apr 19 '11 at 05:59
  • Thanks :) Can you update your answer with some relevant links, especially to docs? Then we can ditch some of the comments. – Jeroen Wiert Pluimers Apr 19 '11 at 10:00
2

You could consider using a higher protocol level like HTTP, because:

  • It's more firewall and VPN friendly;
  • It's well documented and known as a good protocol;
  • It already has secured HTTPS version;
  • It has a very low overhead over row TCP/IP;
  • It's ready to use in an AJAX environment (if you need it in the future);
  • Microsoft already did the low-level tuning for you in modern version of Windows.

In this case, you could take a look at two Open Source classes working from Delphi 6 up to XE:

THttpApiServer which implements a HTTP server using fast http.sys kernel-mode server:

  • The HTTP Server API enables applications to communicate over HTTP without using Microsoft Internet Information Server (IIS). Applications can register to receive HTTP requests for particular URLs, receive HTTP requests, and send HTTP responses. The HTTP Server API includes SSL support so that applications can exchange data over secure HTTP connections without IIS. It is also designed to work with I/O completion ports.
  • The HTTP Server API is supported on Windows Server 2003 operating systems and on Windows XP with Service Pack 2 (SP2). Be aware that Microsoft IIS 5 running on Windows XP with SP2 is not able to share port 80 with other HTTP applications running simultaneously.

TWinHTTP which handles client-side HTTP/1.1 request using the WinHTTP API:

  • Microsoft Windows HTTP Services (WinHTTP) is targeted at middle-tier and back-end server applications that require access to an HTTP client stack;
  • Is much faster than older WinINet API.

Resulting speed is very good (especially the server), and you will rely on Microsoft implementation. The first is the core of IIS, and the second is used in the latest versions of Internet Explorer.

Arnaud Bouchez
  • 42,305
  • 3
  • 71
  • 159
  • In this particular case, the protocol that runs on top of TCP/IP needs to be robust, light weight and connection based (not session based), the server can be either Windows or non-Windows, and the protocol should support authentication by a non-Windows server. Upon termination of the connection, the server should release any back-end resources. If you have suggestions for a good base protocol, please let me know. – Jeroen Wiert Pluimers Apr 19 '11 at 10:37
  • HTTP/1.1 can be connection based AFAIK - you just need to keep the connection alive - and support authentication by a non-Windows server or client. But it's better oriented into a stateless architecture, like REST. – Arnaud Bouchez Apr 19 '11 at 13:40
1

Worked with NetMaster components way (way!) back in the old Delphi versions (2! 3! 4!)

Did some work with Indy, but had the unnatural feeling also (actually I'd describe it more as bulky)

Stumbled upon Synapse when I was searching for just a light wrapper around the Windows network API,

And then rediscovered plain old TTcpClient/TTcpServer. They are Delphi's own wrapper around winsock! I use them blocking, with a dedicated TThread inheritant for each TTcpClient, and let TTcpServer do the threads and do all the work in DoAccept, see here for an example.

This, fow now, gave me the rock-solid feel we're looking for. If you want to support heavy load, I would try and build a thread manager that handles several sockets/connections per thread, or have two sets of threads: a few that listen a larger number of 'dormant' connections, and the others that handle lesser 'active' connections, switching connections between the threads depending on wether a request or response is being handled. (e.g. HTTP's Connection: keep-alive)

Stijn Sanders
  • 35,982
  • 11
  • 45
  • 67
  • 4
    As far as I remember, there are some issues in TTCPClient/TTCPServer implementation. See what Remy wrote in https://forums.embarcadero.com/thread.jspa?threadID=44767 "TTcpClient (and TTcpServer) is a HORRIBLY written component. It was Borland's attempt at a cross-platform socket solution for the CLX framework, but they wrote it in a least-common-demoninator manner that ends up with very little useful functionality. I strongly advice you to stop using it altogether and either use the legacy VCL TClientSocket component, or switch to a third-party socket library, such as Indy, ICS, or Synapse." – Arnaud Bouchez Apr 19 '11 at 09:57