How to execute "Parallel.ForEach" as background task that will return control immediate to the calling method?

Question

How to execute "Parallel.ForEach" as background task that will return control immediate to the calling method?

I want to using C#/Linq "Parallel.Foreach" but I don't want to way until all parallel tasks complete before proceeding to the next statement in the call method. I'm looking for a way to handle the completation of "Parallel.Foreach" asynchronously from another thread.

using System.Threading.Tasks;
using System.Collections.Concurrent;

public class stuff {    
    int[]                jobs;
    ConcurrentQueue<int> done;

    void DoSomething(int job_id) {
       //This takes a long time...
       Sleep(10000);
    }

    void Button_Click() {
        jobs = new int[] {0,20,10,13,12};

        // parallel.foreach...ok great...now all CPUs will 
        // be loaded in parallel with tasks... 
        // … But, How to get this to return 
        // immediately without awaiting completion?
        Parallel.ForEach(jobs, job_i => {
            DoSomething(job_i);
            done.Enqueue(job_i); //tell other thread that job is done
        });
        // now my gui is blocked and unresponsive until this 
        // foreach fully completes...instead, 
        // I want this running in background and returning immediately to gui
        // so that its not hung...
    }
 }

Answer 1

Well for starters I would recommend an async void EventHandler.

An async eventhandler with an await method will prevent the UI/main thread from being frozen. To benefit from said asynchronous-ness you will need an async/await in the code.

Propose change to this:

void Button_Click() {
    jobs = new int[] {0,20,10,13,12};

    // parallel.foreach...ok great...now all CPUs will 
    // be loaded in parallel with tasks... 
    // … But, How to get this to return 
    // immediately without awaiting completion?
    Parallel.ForEach(jobs, job_i => {
        DoSomething(job_i);
        done.Enqueue(job_i); //tell other thread that job is done
    });
    // now my gui is blocked and unresponsive until this 
    // foreach fully completes...instead, 
    // I want this running in background and returning immediately to gui
    // so that its not hung...
}

To This:

    public async void Button_Click()
    {
        await DoMyWorkAsync();
    }

    private async Task DoMyWorkAsync()
    {
        await Task.Run(() =>
        {
            jobs = new int[] { 0, 20, 10, 13, 12 };

            Parallel.ForEach(jobs, job_i =>
            {
                DoSomething(job_i);
                done.Enqueue(job_i);
            });
        });
    }

Note: There maybe other considerations to be careful of, for example, someone double clicking. However, to answer the original question - this should be all the change you need. See below for a quick and dirty ThreadSafeBoolean.

For TAP/C# Purists:
If Toub, Cleary, or Skeet, were here they would caution me against the await Task wrapper - however I have found that in the real world to maintain Async/Await patterns - this pattern is occasionally needed. Even though Parallel supports async lambdas too the behavior is woefully unpredictable. You would need something like NitroEx or ForEachAsync extenstions by Yuri. However, if you want to run something like these synchronous fire and forgets in parallel and asynchronously - async Task wrapper is usually the simplest solution.

Below I demonstrate handling double clicks with a thread safe boolean backed by Interlocked. For more readable / generalized code I would also consider if (Monitor.TryEnter(myObjectLock, 0)) or a SemaphoreSlim(1,1). Each one has a different usage style. If I was sticking with a pattern for this case then Monitor.TryEnter maybe the cleanest approach as it will return false and you can exit out similar to the thread safe bool. For efficiency and simplicity I assume we are to just pass through the Event (i.e. do nothing) if its already running.

Sample Busy Check With Threadsafe Boolean:

using System.Threading;

// default is false, set 1 for true.
private static int _threadSafeBoolBackValue = 0;

public bool ThreadSafeBusy
{
    get { return (Interlocked.CompareExchange(ref _threadSafeBoolBackValue, 1, 1) == 1); }
    set
    {
        if (value) Interlocked.CompareExchange(ref _threadSafeBoolBackValue, 1, 0);
        else Interlocked.CompareExchange(ref _threadSafeBoolBackValue, 0, 1);
    }
}

public async void Button_Click(object sender, EventArgs e)
{
    if (!ThreadSafeBusy)
    {
        ThreadSafeBusy = true;
        await DoMyWorkAsync();
        ThreadSafeBusy = false;
    }
}

private async Task DoMyWorkAsync()
{
    await Task.Run(() =>
    {
        jobs = new int[] { 0, 20, 10, 13, 12 };

        Parallel.ForEach(jobs, job_i =>
        {
            DoSomething(job_i);
            done.Enqueue(job_i);
        });
    });
}

For other performance optimizations, consider alternatives to Concurrent inserts and if the workload is intense, limit the parallel foreachs processed concurrently to new ParallelOptions {MaxDegreeOfParallelism = Environment.ProcessorCount}.

private async Task DoMyWorkAsync()
{
    await Task.Run(() =>
    {
        jobs = new int[] { 0, 20, 10, 13, 12 };

        Parallel.ForEach(
            parallelOptions: new ParallelOptions { MaxDegreeOfParallelism = Environment.ProcessorCount },
            source: jobs,
            body: job_i =>
            {
                DoSomething(job_i);
                done.Enqueue(job_i);
            });
    });
}

Answer 2

You can run Parallel.ForEach in seprate task and call a delegate when all tasks done :

int[] jobs;
ConcurrentQueue<int> done = new  ConcurrentQueue<int>();

jobs = new int[] {0, 20, 10, 13, 12};

//How to get this to return immediately without awaiting completion?
Task tsk = new TaskFactory().StartNew(() =>
    {
        Parallel.ForEach(jobs, jobI =>
        {
            DoSomething(jobI);

            done.Enqueue(jobI);
        });
    })
    .ContinueWith(task =>
    {
        MessageBox.Show("Parallel.ForEach complete");
    });

Answer 3

Method 1:

var thread = new Thread(() => {
    //JobInit();
    Parallel.ForEach(jobs, job_i => {
        DoSomething(job_i);
        done.Enqueue(job_i);
    });
    //stop = true;
});

thread.Start();
//stop = false;

Method 2:

bool run_task() {        
    Parallel.ForEach(jobs, job_i => {
        DoSomething(job_i);
        done.Enqueue(job_i);
    });
    return true;
}

void Button_click() {

    // save task variable to run in background
    Task.Run(() => { run_task() });
}

Notes:

(but, I found that the overhead that parallel.foreach inserts for each task it starts is quite high for some reason.. seems like it starts too many worker tasks at the same time and they over compete for limited resouces on the same cpu...ends up being slower then another case where I started 4 tasks in parallel and then waited on them all to complete before getting another batch of 4 parallel tasks...that usage case of limiting parallel batches of 4 task each time when going through a large array was much faster...)