0

I have around 50 similar tasks - each one performs network calls to some server and writes the response to the db. Each task has its own set of calls it should make. Each task talks to one server, if that's relevant. I can have one process, each running its own task with its own set of calls it should make (so no memory sharing is needed between the threads). Another option is a process for each task. Or a combination. Should I choose multiple threads because switching between processes is more costly? It runs on Ubuntu. I don't care about the cost of the creation of the thread/process. I want to start them and let them run for a long time.

Rony Tesler
  • 1,207
  • 15
  • 25
  • 2
    Process creation is way more costly than creating threads – πάντα ῥεῖ Jan 13 '23 at 19:03
  • Try it and see, very difficult to say in general. There isn't much difference between a thread and a process as far as the kernel is concerned – Alan Birtles Jan 13 '23 at 19:03
  • Too many unknown variables to say. But in general, processes are more expensive to start than threads, but over the long haul the start-up costs will fade away. Unless you're constantly creating new processes or threads. If that's the case consider pooling. – user4581301 Jan 13 '23 at 19:05
  • @πάνταῥεῖ I edited the question and made it more clear about ignoring the creation cost. – Rony Tesler Jan 13 '23 at 19:16
  • @user4581301 What else is unknown? I'll add the missing information. – Rony Tesler Jan 13 '23 at 19:19
  • 1
    My **guess**, in terms of performance, is that using threads will trigger less cache misses since probably many data is shared. So threads could be faster, but it all depends on the size of used memory, the cache size in CPU, etc. – prapin Jan 13 '23 at 19:27
  • 1
    ^ a few of the unknowns. Personal opinion: Unless you need to securely wall off the tasks from one another and have a strong case for killing tasks, start with the threaded approach. – user4581301 Jan 13 '23 at 19:29
  • 1
    @user4581301 Multi-process is the "unix way". The entire Unix philosophy is built around individual processes cooperating to build larger systems. Keep that in mind. – Something Something Jan 13 '23 at 19:31
  • 3
    In this case the runtime performance is going to be about the same. Go for processes - a lot simpler and more reliable. So one bad task will crash only one process and not all of them. – Sergei Jan 13 '23 at 19:32
  • What you may find is these 23 tasks work well as one multi-threaded process (or even one thread with a statemachine or synchronizer like `select` or overlapped IO) and 4 other tasks make a good threaded process and the others work best as totally independent processes. – user4581301 Jan 13 '23 at 19:32
  • @NoleKsum I can't. I already totally forgot about do one thing and do it well. – user4581301 Jan 13 '23 at 19:33
  • 1
    But seriously, the question is about which is faster. Threads will probably be faster, but minimally and not as safe. Or easy to write and maintain. – user4581301 Jan 13 '23 at 19:34
  • @prapin As I said, no need to share data between the threads. Only the connection to the db, but I don't think it's a problem to create one for each task. It's influxdb, I don't think it's an open connection, but I need to check. – Rony Tesler Jan 13 '23 at 19:35
  • 2
    At the end of the day, profile. Write both and see which one really is faster. You'll probably find separate processes is fast enough, and if it's easier to manage, and it usually is, it's the winning plan, speed be damned. – user4581301 Jan 13 '23 at 19:38
  • ***each one performs network calls to some server and writes the response to the db.*** Are the tasks completely independent of each other? If so maybe you have a single command line program and have your OS utilities to do the batch work for you. Maybe you want this: [https://www.gnu.org/software/parallel/parallel_tutorial.html](https://www.gnu.org/software/parallel/parallel_tutorial.html) – drescherjm Jan 13 '23 at 20:02
  • 3
    Network latency almost certainly swamps any other performance differences. – Pete Becker Jan 13 '23 at 20:19
  • 1
    Comparing apples and oranges. Processes are isolated from each other at OS level and threads aren't. Yes threads can cooperate with each other faster, but if there's a crash they all go down together. So it's more a question of safety than performance. You can use both at the same time BTW. – rustyx Jan 13 '23 at 20:38
  • 1
    In theory, for this kind of task (wait on network connection and wait on database queries), the best in terms of performance and memory usage is to use **coroutines**. C++20 introduced native low-level support for coroutines in the core language, but they are hard to use, unless you found a suitable library. Since coroutines require no context switch and much less cache misses, they can be used to support thousands of concurrent connections easily. – prapin Jan 14 '23 at 14:58

1 Answers1

3

Strictly in terms of performance, it comes down to initialization time because as far as execution time goes, in this particular case that there is no interprocess communication required, execution time will be very similar.

Threads are faster to initialize since there is less duplication of resources to initialize.

The so much advertised fact that "threads switch faster than process" like in this StackOverflow question/answer is a preposterous non-sequitur. If a batch is running at 100% cpu and there are enough free cores to supply the demand, the OS will not ever bother to switch threads into new cores. This will only happen when threads block and wait for input (or output chance) from network or disk - in which case the price of 3-20 microseconds will completely overshadow the few nanoseconds of advantage of an additional memory cache miss.

The only reason why the operating system would preempt a thread or process other than I/O block is when the machine is running out of computing resources, ie when it is running at more than 100% on every single available CPU. But this is such a special, degenerated case that should not belong in a generic response.

However if the running time is much larger than the initialization time, say the process runs for several minutes while it takes a few milliseconds to start up, then performance should definitely be the same and your requirement should be other like facility of administration of the process batch. Another point to keep in mind is that if one process dies, the others finish, while if one thread crashes, the entire batch dies as well.

There might be some extra benefits of using threads as well due to the lesser use of system resources but I believe these would be negligible.

Something Something
  • 3,999
  • 1
  • 6
  • 21
  • 1
    There are also a few things, like killing them and isolating memory, that you can do with processes and you can't do safely with threads. – user4581301 Jan 13 '23 at 19:08
  • @user4581301 Yep, true. If performance gain is marginal, these admin tasks should definitely be taken in consideration. – Something Something Jan 13 '23 at 19:09
  • But what about the process switch vs the thread context switch? Isn't the process switch more costly because you also need to switch the virtual memory? – Rony Tesler Jan 13 '23 at 19:18
  • @RonyTesler What do you mean by "switch"? – Something Something Jan 13 '23 at 19:19
  • @RonyTesler I am pretty sure that the context switch done by the OS is the same between threads and processes. Linux threads are actually implemented as shared-memory processes. – prapin Jan 13 '23 at 19:24
  • 1
    @NoleKsum I read the accepted answer here - https://stackoverflow.com/questions/5440128/thread-context-switch-vs-process-context-switch – Rony Tesler Jan 13 '23 at 19:25
  • @prapin What do you mean "context switch"? Calling into the kernel? – Something Something Jan 13 '23 at 19:26
  • @RonyTesler That answer makes no sense. Threads do not "switch" like that. If you have several threads or processes running one on each core on an otherwise empty machine, there will be no switches other than system calls into the kernel. In that case the cost is exactly the same. That question and respective answer makes absolutely no sense. – Something Something Jan 13 '23 at 19:28
  • 1
    @NoleKsum The context switch is the operation done by the OS, inside an interrupt handler to save the thread context (mostly CPU registers) into memory, and restore the context of another thread from memory. – prapin Jan 13 '23 at 19:30
  • @prapin Why would the OS switch threads in first place? That should never happen, unless the machine is running over capacity. – Something Something Jan 13 '23 at 19:32