10

I need to develop a real-time (i.e., info is requested and received at least once per second) monitoring application in Delphi, which monitors multiple remote devices (can be hundreds). The communication is via TCP/IP.

I am looking for advice to develop this app avoiding 100% CPU consumption and minimizing the amount of RAM used. In other words, I want my application to remain responsive and not block the system or consume all resources.

My main concern is about using threads for monitoring each remote device. Is there a limit to the number of threads my application can create? Can the threads can be started using a low or medium priority to minimize the CPU consumption?

Advice about optimal memory usage is welcome, too.

Lieven Keersmaekers
  • 57,207
  • 13
  • 112
  • 146
Salvador
  • 16,132
  • 33
  • 143
  • 245
  • Your main concern should be how much effort is involved to implement this in a robust manner ;-) Having done this all before, CPU and RAM usage are the least of your concerns and will not be an issue if you have a well architected solution. – Misha May 03 '11 at 23:23

4 Answers4

3

Your instinct is right, you want to handle the logging in threads outside of the main thread. Create a simple tcp/ip server that creates a new thread for the incoming connection and handles the logging there. Obviously you'll want to do things like track each thread to be able to terminate it when your server application closes, and possibly implement a thread pool/queue if you want to re-use them rather than constantly creating and destroying them. But what you describe is actually a fairly simple server application in concept. There's no hard and fast limit to the # of threads you can create. But unless the connections are constant and kept open, you may be suprised how few actually get created simultaneously.

GrandmasterB
  • 3,396
  • 1
  • 23
  • 22
  • this doesn't sound like good advice to me. I don't see what problem threads are solving here. I would also point out that large numbers of threads cost address space for their stacks even if this is actually memory backed by page file. – David Heffernan May 03 '11 at 22:51
  • 1
    performing the logging in a separate thread will prevent the main application from being blocked (which was one of the requirements of the poster) if the operation they are performing is time consuming. Also, imo, the model is easier to manage, encapsulate, and more scalable. The actual impact of creating the threads on a modern machine would be trivial. This is what the Apache web server does, for example. – GrandmasterB May 04 '11 at 05:07
  • @David, a thread is not at all expensive! Your comment prompted me to run an experiment: http://pastebin.com/Drvmi2MH I ran 1000 threads on a single-core Windows XP machine; The CPU consumption was at around 50% and the application had a 10Mb working set. If the OP's threads do TCP/IP, they'll be mostly waiting for I/O, so the CPU consumption would actually be better then mine. – Cosmin Prund May 04 '11 at 08:06
  • 1
    I have 1000 threads running in a thread pool (idling) in an application developed using my framework on a single core Windows XP VM machine. The average CPU usage is around 4% and the amount of RAM used is around 20MB. – Misha May 04 '11 at 08:55
  • @cosmin @misha The expense is address space as I said in my comment. – David Heffernan May 04 '11 at 11:36
  • @David, so why does this matter? – Misha May 04 '11 at 12:00
  • @Misha You have a finite budget of address space. It's best not to use it all up! I simply see no reason to create large numbers of threads for this application. – David Heffernan May 04 '11 at 12:03
  • @David, while I do not see any reason why you would use it if it is not needed, I do not see why you cannot use as much as you require? To go back to the TCP server issue, one thread per connection is the way that Indy implements a TCP server, and this will handle in the range of 100-1000 connections. More than that and you need another architecture. So the response to the original question is, one thread per connection will do just fine. – Misha May 04 '11 at 12:18
  • @Misha each thread has its own stack whose memory is reserved when the thread is created. – David Heffernan May 04 '11 at 12:20
  • @David, our threads are much, much smaller then your threads. (thinking of [this](http://stackoverflow.com/questions/5671052/delphi-64-bits/5671955#5671955)). The default stack size for a new thread appears to be 1Mb. With 1000 threads that's not going to cause significant address-space depletion (it leaves 50% address space for the rest of the app), and 1000 shouldn't be used any way; My 1000 was for testing purposes. – Cosmin Prund May 04 '11 at 12:46
  • You can change the default stack size. The default of 1MB is overly generous, unless you do a lot of recursion. – Caleb Hattingh May 04 '11 at 13:03
1

Good luck doing this yourself. I have invested a couple of years to develop a generic platform to do just this. If you want you can have a look at it here. At the very least you will need to use some comms components that already work very reliably, such as RemObjects.

Misha
  • 1,816
  • 1
  • 13
  • 16
  • 2
    A tcp/ip server is not a complex program to create. He doesnt need 'luck', just a few hours of research. – GrandmasterB May 04 '11 at 05:12
  • Aah, yes, but having something that works 24/7 with lots of connections across dodgy networks without skipping a beat is a different beast altogether (production versus prototype). Including both client and server I have more than 5,000 lines of code dedicated to the TCP functionality to ensure that everything works every time (and this is built on the Indy components). Of course, if you use something like RemObjects, they have already done this for you. – Misha May 04 '11 at 07:03
  • Here are some issues to consider (not all might be required): message protocols, message parsing, message encoding, data compression, data encryption, client pings, reconnection on failure, message broadcasting, orderly application startup, orderly application shutdown, inter-thread messaging, request/response patterns, notification/one-way messages, send delays and Nagle's algorithm, threaded error handling, message validation, message prioritisation, chunking large messages into smaller packets, message timeouts, debug message logging, comms channel statistics, thread CPU logging – Misha May 04 '11 at 07:37
  • 1
    @Misha, you're making this sound much worst then it actually is. I'm with GrandmasterB on this one: it's doable, but requires some networking knowledge. Glancing over to the screen on my left I can see my "RC Proxy" server written with Delphi 7 + ICS handling 47 simultaneous connections, over two ISP connections, and it pretty much runs 24/7. – Cosmin Prund May 04 '11 at 08:13
  • @Cosmin, I would be interested to know how much effort you put into that system over the entire lifetime of the system. IMHO, when things are developed over a period of time the amount of effort (and knowledge required) is vastly underestimated. Why don't you send me your code and we can compare notes? – Misha May 04 '11 at 08:25
  • I'll put it another way - unless there is a real need to do this from scratch yourself, I wwould strongly recommend starting with a suite of inter-process messaging components because the amount of effort involved is significantly more than a few hours of research and a small amount of development. Using something like RemObjects would cut development time down by at least a factor of 10 (and no, I don't use RemObjects)! – Misha May 04 '11 at 08:43
  • @Misha, I dont disagree that there's probably a pre-made solution available here to handle the networking aspect of the OP's server. If nothing else, even Indy or Synapse could be used to simplify development. But I do disagree about the difficulty of creating a tcp/ip server that runs 24/7. Some of it of course depends on the specific nature of how the OP wants to handle connections. But something like a web server that receieves and dispatches is not difficult at all after researching a bit on how to use sockets. – GrandmasterB May 04 '11 at 18:05
  • There is a significant amount of code in a web server, which kind of proves my point (the Indy code to implement a simple HTTP client/server is around 5,000 lines). Start from higher up the chain and it is not that much work. Start from lower, e.g. Indy, and it is more work than you think. – Misha May 04 '11 at 22:31
1

I would not use a thread for each device, I'd investigate if a thread can serve a pool of devices, or if a thread pool can be used to dispatch incoming data to the first available thread.

You can check this article about process and thread limits in Windows. Threads can have lower priority, but beware then they can be pre-empted by threads with higher priority, and not being able to read/write data in time. Also too many threads could simply "waste" time if they have nothing to do but the scheduler is forced to give them a slice of time (the thread should be very well behaved to avoid just using it to do nothing). How many threads could be run concurrently without issues strictly depends on the available hardware.

A lot depends on how the data is delivered (is pull or push? have all the data equal priority, or not?) and the process effort once data are read, and how fast the system should react to new data. For example one solution could involve reading data and queueing them for processing, but it couldn't work if the response time should be shorter than the queue could allow.

1

I would question if you really need threads at all. Best code is the code you don't have to write.

Async - non blocking, event based communication (using ICS, Clever, etc) would be much easier to implement.

What you describe is a client application, threads are generally required for servers. Also 'once per second' is not that frequent even with hundreds of connections.