How to run two tasks in parallel with C#?

Question

Suppose that I have this code that should be run two tasks correctly in parallel , I didn't have a good idea about parallels tasks in C# and I would like to just start with this code to understand this concept, what I would like to do is running two task in the same time ( asynchronously )

 public async Task Main() {
     var task1 = Task.Run(() => DoWork());
     var task2 = Task.Run(() => CleanIt());

     await Task.WhenAll(task1, task2);
 }

 private void CleanIt() {
     int sum = 0;
     for (int i = 0; i < 10; i++) {
         Console.WriteLine(" Thread two " + i);
     }
 }

 private void DoWork() {
     int sum = 0;
     for (int i = 0; i < 10; i++) {
         Console.WriteLine(" Thread one " + i);
     }
 }

The result that I've got:

 Thread one 0
 Thread two 0
 Thread one 1
 Thread one 2
 Thread one 3
 Thread one 4
 Thread one 5
 Thread one 6
 Thread one 7
 Thread one 8
 Thread one 9
 Thread two 1
 Thread two 2
 Thread two 3
 Thread two 4
 Thread two 5
 Thread two 6
 Thread two 7
 Thread two 8
 Thread two 9

I would like to show the result like this :

 Thread one 0
 Thread two 0
 Thread one 1
 Thread two 1
 Thread one 2
 Thread two 2
 ....

How can I achieve this result ?

If you want things to happen in a particular order you need to somehow enforce that order. Right now there is nothing connecting these two tasks, you will need some kind of barrier — UnholySheep, May 10 '22 at 15:17
Simplest would be to make the methods accept a parameter for the index they should work with, and then run the two methods in parallel for index 0, then wait for them both to complete, then run both in parallel for index 1, wait, index 2, wait, etc. — Lasse V. Karlsen, May 10 '22 at 15:20
Yes, as you know it's just an example to understand the concept of Threading in C# , In fact, I would like to understand how multithreading works , also I would like to understand why the second task wait for the first task to complete , ( maybe I'm not correct ) but as what I see, this is what is understood . — abdou_dev, May 10 '22 at 15:24
You have to write some kind of synchronization system. You need some shared variable which can synchronize the operations. Keep in mind that you have to manage the access to that shared resource in order not lo let the 2 tasks to interfer each other. Furher in your case the 2 task looks like to run one after the other but it's just a case... you cannot be sure of the order if you don0t control it yourself — Marco Beninca, May 10 '22 at 15:25
The reason you have all of task 1 finish before task 2 is run is that task 1 is very fast. it is done before the relatively expensive startup of task 2 is finished. In order to simulate longer running tasks you can let them sleep a random amount of time in each iteration (some tenths of a second, which is long compared to task startup times). You'll see that they print in random order. Oh, and please rename `CleanIt` to `DoWorkToo` or so. It doesn't clean, does it. The naming hides the purpose of your program -- I thought you are really cleaning up what task 1 left behind. — Peter - Reinstate Monica, May 10 '22 at 15:27
@MarcoBeninca , this is not related to the main difference between Asynchronous vs synchronous `When you execute something synchronously, you wait for it to finish before moving on to another task. When you execute something asynchronously, you can move on to another task before it finishes.` https://stackoverflow.com/questions/748175/asynchronous-vs-synchronous-execution-what-is-the-main-difference — abdou_dev, May 10 '22 at 15:28
@Peter-ReinstateMonica, that was a good point , so why the first task is very fast??? , as you can see they have the same pieces of code , why the first is fast that the second , I would like to know what's the reason of this? — abdou_dev, May 10 '22 at 15:30
Do you know beforehand that the `CleanIt` and `DoWork` tasks will both have an equal number of internal step? Like 10 steps as in the example? — Theodor Zoulias, May 10 '22 at 15:33
The second task is very fast, too ;-), so if you start a third task it's likely starting only after the second is already finished, and so on. But the reason that the second one starts only after the first one is finished is that the *first* one is fast. How fast the second one is is irrelevant. — Peter - Reinstate Monica, May 10 '22 at 15:33
@Peter-ReinstateMonica, why it starts after the second??? why they will not start in the same time ??? that was my question? — abdou_dev, May 10 '22 at 15:35
Are you interested for an answer that will show how to install a mechanism that will synchronize these two tasks, so that their internal steps are always executed in pairs? In this case, is it OK if the answer requires the installation of a third-party package? — Theodor Zoulias, May 10 '22 at 15:41
@TheodorZoulias , Yes I'm searching for a solution to run two tasks in parallels ( no matter what's the method or with install a third-party package ). — abdou_dev, May 10 '22 at 15:44
all you can do is *tell the processor* that it can run either of these threads at any time. when the processor decides to schedule each thread on a CPU is not in your control. It can decide to run one thread to completion, then the other. Or it can decide to run them both on different cores. Or it can switch between them. That's part of the OS, not your code. — Esther, May 10 '22 at 15:47
Sorry for asking too many questions, but is it OK if the solution requires that the code of the `CleanIt` and `DoWork` methods is modified? Or you expect a solution that will synchronize the two methods externally, without touching their internals? — Theodor Zoulias, May 10 '22 at 15:48
the very definition of "asynchronously" means that you *don't* control when each task gets done, but rather the OS's processor scheduling mechanism does. If you create very long-running tasks, you are more likely to see it switching between them, but that is still not in your control. And every time you run the code, you'll see them in a different order. — Esther, May 10 '22 at 15:49
@Esther , So the threads in the case of asynchronous is out of my controls ?? — abdou_dev, May 10 '22 at 15:56
@TheodorZoulias , Actually , I would like a method that will synchronize the two methods externally , without touching their internals ( without touching the pieces of codes ) — abdou_dev, May 10 '22 at 15:56
I see. FYI I don't know of a solution that satisfies this requirement. :-) — Theodor Zoulias, May 10 '22 at 16:02
@abdou_dev what is your *actual* problem? It's not to "synchronize" two methods, whatever that means. Tasks don't work the way you assumed either. 10 iterations on a 3GHz machine is nothing so it's quite possible one of these tiny tasks completes before the other. If you used eg 100K iterations you'd see different results. If you added a bit of delay with `await Task.Delay(100)` you'd see different results. Delays aren't a synchronization mechanism though — Panagiotis Kanavos, May 10 '22 at 16:38
@abdou_dev a meaningful question would be to ask how to execute a series of tasks, each of which has a specific continuation. Or how to process a stream of data, eg messages. Or how to have asynchronous producers and consumers. All of these questions have built-in answers, eg using Channels, DataFlow or just running an `async` method that calls two other async methods in a loop — Panagiotis Kanavos, May 10 '22 at 16:40
@PanagiotisKanavos I like your answer and seems that be the logic one compare to others answers. — abdou_dev, May 10 '22 at 16:55
@abdou_dev "this is not related to the main difference between Asynchronous vs synchronous" - indeed. The problem is you think you've asked about something (I don't get what you think it is by reading the question/comments) but what is *written* in the question is "how to *synchronize* code running concurrently". Please note that "synchronize" as in "coordinate execution in some way" is unrelated to "synchronous" as opposed to "async". — Alexei Levenkov, May 10 '22 at 18:14

Esther · Answer 1 · 2022-05-10T16:14:09.137

1

The idea of async code is that you tell the processor not to wait for this task to be completed, but rather start other tasks meanwhile. Since the "other" tasks don't wait for the original task to be completed either, there is no way to ensure that the tasks stay in sync without writing synchronous code. When you start multiple async tasks, you give up control of exactly when those tasks are executed to the OS and its processor scheduling algorithm, with the promise that they will be done eventually.

The scheduling algorithm will choose the most efficient way of executing your code, taking into consideration every other program that is asking for CPU time, and decide how to run your tasks. It can choose to run one task to completion, and then the other task. Or, it can choose to run them on different processor cores, or switch between them on the same core. This is not up to you, but rather up to the OS to decide.

Specifically, in C#, async tasks are run using a thread pool so you can't even choose whether or not to run multiple threads. If the tasks are run using the same thread, they can't run on multiple cores, which means that they can't run in parallel. If you want tasks to run on multiple cores, you need to use Threads explicity, but regardless you can't control when each thread runs or which core it runs on.

If you have long-running tasks, you may see that your output switches from one to the other. The exact amount of time that each task runs for depends on many things that you can't control, and will possibly be different each time you run the program.

If you need your code to run synchronously (ie, waiting for some other task to continue before running a task), you need to write synchronous code, not async.

edited May 10 '22 at 16:14

answered May 10 '22 at 16:00

Esther

450
3
9

@TheodorZoulias I fixed that part. tasks can sometimes be run in multiple threads, but won't necessarily be unless you do that explicitly – Esther May 10 '22 at 16:14
By *"async tasks"* do you refer to `Task`s created by `async` methods, or to `Task`s created by the `Task.Run` method? – Theodor Zoulias May 10 '22 at 16:20
@TheodorZoulias tasks created by Task.Run – Esther May 10 '22 at 16:22
1

OK. So if you start two tasks with the `Task.Run` method at the same time on a multi-core machine, and the two tasks are doing non-trivial work (let's say that each one is doing some CPU-bound calculation that lasts 1 second), do you think that it's more likely that the two tasks will run sequentially on the same CPU core, or that they will run in parallel on two different cores? Or that both cases are equally likely (fifty-fifty)? – Theodor Zoulias May 10 '22 at 16:30
@TheodorZoulias I would assume that running them in parallel is most efficient here, but I am not the task scheduler :) and even if they are run on different threads, whether to run them in parallel is up to the OS and depends on what else is running. – Esther May 10 '22 at 16:31
True, you are not a scheduler, but what are your expectations? If you expect that both tasks will most likely run sequentially, why would you bother messing with tasks in the first place? You could just run the two methods sequentially, and call it a day! :-) – Theodor Zoulias May 10 '22 at 16:35
@TheodorZoulias https://stackoverflow.com/questions/25591848/async-await-multi-core – Esther May 10 '22 at 16:39
[That](https://stackoverflow.com/questions/25591848/async-await-multi-core) question is about `async`/`await`. Your answer refers to tasks created with `Task.Run` (according to [this](https://stackoverflow.com/questions/72188870/how-to-run-two-tasks-in-parallel-with-c/72189591?noredirect=1#comment127547013_72189591) comment). – Theodor Zoulias May 10 '22 at 17:27
@TheodorZoulias that is true, I was overly hasty with that link. But my answer still stands: you aren't controlling when or where tasks get run, all you have is a promise that they will get done eventually. when and where is up to the process or the OS. – Esther May 10 '22 at 17:54
Esther I might not be in full control of when exactly my tasks will be executed, but I do have some reasonable expectations about it. I **must** have some expectations when I am choosing the tools to use, in order to write a program that has a specified behavior. Otherwise I would be lost in the wind, without knowing where I am coming from and where I am going to. Reading the Microsoft documentation is great, but you have to combine it with some experimentation in order to be able to apply this knowledge in real-life applications. – Theodor Zoulias May 10 '22 at 18:55
@TheodorZoulias yes of course, and if you need specific behavior there are all kinds of options you can use when creating tasks, custom task schedulers you can use, and other ways of insisting things get done a certain way. The default is just to dump the burden of making your tasks run in the most efficient order on the system. – Esther May 10 '22 at 18:59

Peter - Reinstate Monica · Answer 2 · 2022-05-10T17:22:20.860

You do two things sequentially:

(1) start task 1. 
(2) start task 2.

Now starting a task, i.e. executing Task.Run(), is slow and expensive. It takes perhaps, say, 5ms. Running that very short task takes perhaps only 1ms. So task 1 is finished long before task 2 has started.

 
Time ->                 ...1ms...1ms...1ms...1ms...1ms...1ms..........1ms............1ms...1ms...1ms...1ms...1ms...
main [set up task 1..................................][set up task 2............................................][main ends]
task1                                                 [DoWork() starts] [DoWork() ends]
task2                                                                                                            [DoWorkToo() starts] [DoWorkToo() ends]

If you want to have the tasks run in parallel you must make their runtime long in comparison to the task startup time. One solution is simply to run the loop many thousands of times, but that can be unwieldy. Better is to let the tasks sleep during loop execution so that task1 is still running when task 2 starts.

Here is an example with longer running tasks showing nicely how the execution interleaves:

using System;
using System.Threading;
using System.Threading.Tasks;

using System.Diagnostics; // stopwatch

namespace TwoTasks2
{
    class Program
    {
        static Stopwatch globalStopWatch = new Stopwatch();

        private static void DoWork()
        {
            Console.WriteLine("========================= Entering DoWork() after " + globalStopWatch.ElapsedMilliseconds);
            int sum = 0;
            Stopwatch sw = new Stopwatch();
            sw.Start();
            for (int i = 0; i < 100000; i++)
            {
                if (i % 10000 == 0) 
                { 
                    Console.WriteLine("Task one " + 10000 + " cycles took " + sw.ElapsedMilliseconds + " ms");
                    sw.Stop();
                    sw.Start();
                    Console.WriteLine("Thread ID: " + Thread.CurrentThread.ManagedThreadId);
                    Console.WriteLine("CPU ID:    " + Thread.GetCurrentProcessorId());
                }
            }
        }

        private static void DoWorkToo()
        {
            Console.WriteLine("========================= Entering DoWorkToo() after " + globalStopWatch.ElapsedMilliseconds);
            int sum = 0;
            Stopwatch sw = new Stopwatch();
            sw.Start();
            for (int i = 0; i < 100000; i++)
            {
                if (i % 10000 == 0) 
                {
                    Console.WriteLine("                                  Task two " + 10000 + " cycles took " + sw.ElapsedMilliseconds + " ms");
                    sw.Stop();
                    sw.Start();
                    Console.WriteLine("                                  Thread ID: " + Thread.CurrentThread.ManagedThreadId);
                    Console.WriteLine("                                  CPU ID:    " + Thread.GetCurrentProcessorId());
                    
                }
            }
        }

        public static void Main()
        {
            globalStopWatch.Start();
            var task1 = Task.Run(() => DoWork());
            long ms = globalStopWatch.ElapsedMilliseconds;
            Console.WriteLine("--------------------- RunTask 1 took " + ms);

            var task2 = Task.Run(() => DoWorkToo());
            Console.WriteLine("--------------------- RunTask 2 took " + (globalStopWatch.ElapsedMilliseconds-ms));

            var tasks = new Task[] { task1, task2 };
            Task.WaitAll(tasks);
        }
    }
}

Example output on my machine, Debug build:

--------------------- RunTask 1 took 23
========================= Entering DoWork() after 39
--------------------- RunTask 2 took 18
Task one 10000 cycles took 0 ms
Thread ID: 4
========================= Entering DoWorkToo() after 41
                                  Task two 10000 cycles took 0 ms
                                  Thread ID: 5
                                  CPU ID:    1
CPU ID:    2
                                  Task two 10000 cycles took 1 ms
                                  Thread ID: 5
Task one 10000 cycles took 2 ms
Thread ID: 4
CPU ID:    4
                                  CPU ID:    1
Task one 10000 cycles took 2 ms
Thread ID: 4
CPU ID:    4
                                  Task two 10000 cycles took 2 ms
                                  Thread ID: 5
Task one 10000 cycles took 2 ms
Thread ID: 4
CPU ID:    4
                                  CPU ID:    1
Task one 10000 cycles took 2 ms
Thread ID: 4
CPU ID:    4
                                  Task two 10000 cycles took 2 ms
                                  Thread ID: 5
                                  CPU ID:    1
Task one 10000 cycles took 2 ms
Thread ID: 4
CPU ID:    2
                                  Task two 10000 cycles took 2 ms
                                  Thread ID: 5
                                  CPU ID:    1
Task one 10000 cycles took 3 ms
Thread ID: 4
CPU ID:    2
                                  Task two 10000 cycles took 2 ms
                                  Thread ID: 5
                                  CPU ID:    1
Task one 10000 cycles took 3 ms
Thread ID: 4
CPU ID:    2
                                  Task two 10000 cycles took 2 ms
                                  Thread ID: 5
                                  CPU ID:    1
Task one 10000 cycles took 3 ms
Thread ID: 4
CPU ID:    2
                                  Task two 10000 cycles took 2 ms
                                  Thread ID: 5
                                  CPU ID:    3
Task one 10000 cycles took 3 ms
Thread ID: 4
CPU ID:    4
                                  Task two 10000 cycles took 2 ms
                                  Thread ID: 5
                                  CPU ID:    10
                                  Task two 10000 cycles took 3 ms
                                  Thread ID: 5
                                  CPU ID:    1

A Release build is much orderlier:

--------------------- RunTask 1 took 21
========================= Entering DoWork() after 37
--------------------- RunTask 2 took 16
Task one 10000 cycles took 0 ms
Thread ID: 4
CPU ID:    4
Task one 10000 cycles took 0 ms
Thread ID: 4
CPU ID:    6
Task one 10000 cycles took 1 ms
Thread ID: 4
CPU ID:    2
Task one 10000 cycles took 1 ms
Thread ID: 4
CPU ID:    2
Task one 10000 cycles took 1 ms
Thread ID: 4
CPU ID:    3
Task one 10000 cycles took 1 ms
Thread ID: 4
CPU ID:    6
Task one 10000 cycles took 2 ms
Thread ID: 4
CPU ID:    2
========================= Entering DoWorkToo() after 39
                                  Task two 10000 cycles took 0 ms
                                  Thread ID: 5
                                  CPU ID:    11
Task one 10000 cycles took 2 ms
Thread ID: 4
CPU ID:    10
                                  Task two 10000 cycles took 0 ms
                                  Thread ID: 5
                                  CPU ID:    11
Task one 10000 cycles took 2 ms
Thread ID: 4
CPU ID:    10
                                  Task two 10000 cycles took 0 ms
                                  Thread ID: 5
                                  CPU ID:    11
Task one 10000 cycles took 2 ms
Thread ID: 4
CPU ID:    10
                                  Task two 10000 cycles took 0 ms
                                  Thread ID: 5
                                  CPU ID:    1
                                  Task two 10000 cycles took 0 ms
                                  Thread ID: 5
                                  CPU ID:    1
                                  Task two 10000 cycles took 0 ms
                                  Thread ID: 5
                                  CPU ID:    11
                                  Task two 10000 cycles took 0 ms
                                  Thread ID: 5
                                  CPU ID:    11
                                  Task two 10000 cycles took 0 ms
                                  Thread ID: 5
                                  CPU ID:    11
                                  Task two 10000 cycles took 0 ms
                                  Thread ID: 5
                                  CPU ID:    1
                                  Task two 10000 cycles took 1 ms
                                  Thread ID: 5
                                  CPU ID:    1

It takes a whopping 35ms until the first task is running! That's eternity on modern CPUs. The second task then starts much faster.

The tasks take turns even between printing lines to the console. You can also see that even the same thread is hopping from core to core as Windows sees fit. (That actually surprised me. My Ryzen has 6 real cores, and there isn't any significant load, so I'd leave the tasks running where they are.)

Sleeping is a poor test, though, because it explicitly yields control back to the OS. Better to do something that doesn't actually use CPU, but does block and does take a comparatively long time. Say, reading a file using the older synchronous API. — Joel Coehoorn, May 10 '22 at 16:03
this won't make them run in parallel, it will just make it more likely for the OS to do a context-switch. You can't guarantee when either of the tasks will run — Esther, May 10 '22 at 16:04
@Esther no, you cannot guarantee it but you can make it at least possible. And "context switch" can mean different things -- ideally the OS wouldn't be involved when the run time switches between tasks, as opposed to switching between processes. — Peter - Reinstate Monica, May 10 '22 at 16:05
@JoelCoehoorn There is likely no significant difference between blocking and sleeping as far as thread scheduling/reusing is concerned. — Peter - Reinstate Monica, May 10 '22 at 16:14
@Peter-ReinstateMonica there is. Blocking doesn't evict the thread immediately, `Sleep` does. Rescheduling a thread, reloading its stack frame from RAM or disk is so slow that the runtime will try to avoid it when possible. Blocking typically starts with a spinwait before yielding because it's assumed an asynchronous operation won't take a long time. — Panagiotis Kanavos, May 10 '22 at 16:31
@Peter-ReinstateMonica `starting a task, i.e. executing Task.Run(), is slow and expensive` quite the opposite, which is why tasks are used. A Task isn't a thread, it's the specification of a job. In other languages it's called a `promise`. Creating a promise isn't expensive. Promises created with `Task.Run` will execute on a threadpool thread so the cost of starting a new thread is avoided — Panagiotis Kanavos, May 10 '22 at 16:33
Finally, the OP's code doesn't execute tasks sequentially, it fires off two tasks in parallel. The output shows this. The order of the two methods has no effect on how the two threads are scheduled though, on what core they run or how long they use the single `Console` resource. The output clearly shows that `Task2` prints a message shortly after `Task1` — Panagiotis Kanavos, May 10 '22 at 16:36
@PanagiotisKanavos If you read my answer carefully you'll notice that I didn't say the tasks were *executed* sequentially; I said they are *started* sequentially, which is trivially true. And I bet you that starting a task is slow compared to other things a program may do, like writing a string into an output buffer. It is faster than creating an OS thread, if it can use a thread from the thread pool; but don't you think that the thread pool is only *created* when the first task is started? That may be very slow. — Peter - Reinstate Monica, May 10 '22 at 16:43
@PanagiotisKanavos You are also contradicting yourself: "Rescheduling a thread, reloading its stack frame from RAM or disk is so slow..." (which should be at least equally true for setting it up in the first place) vs., answering to my "executing Task.Run() is slow and expensive": "quite the opposite". What now? ;-) — Peter - Reinstate Monica, May 10 '22 at 16:50
There's no contradiction because tasks are promises, not threads. You keep thinking of them as thread which *are* expensive to create. A threadpool though has a number of already active threads, pulling and processing work items from a work queue. Reading from a queue isn't expensive. The worker threads aren't getting rescheduled, so they don't pay the cost of reloading the stack frame. The task itself doesn't have a stack frame. — Panagiotis Kanavos, May 10 '22 at 17:15
@PanagiotisKanavos The first task needs 30+ms to start. The second one is up much faster, so that aligns with your thread/task model. Still, the times are longer than for the short task execution. — Peter - Reinstate Monica, May 10 '22 at 17:27

How to run two tasks in parallel with C#?

2 Answers2