Loop files in a folder and add each to a thread, but only run one of the threads at a time

Question

So I have a folder with several files. I have a loop that will go through each file and add it to a thread to process in the background so the UI is responsive. The problem is I want to only have one thread running at a given time. So basically I want to "queue" the threads and when one completes fire off the next one. What would be the best way to do this? Here is the code I am using. I am wondering if a timer is the best solution? Thanks everyone.

foreach (CustomerFile f in CF)
{
    btnGo.Enabled = false;
    UpdateProgressDelegate showProgress = new UpdateProgressDelegate(UpdateProgress);
    ProcessFile pf = new ProcessFile(this, showProgress, f._FileName, txtDestFolder.Text);
    Thread t = new Thread(new ThreadStart(pf.DoWork));
    t.IsBackground = true; 
    t.Start();
}

http://stackoverflow.com/questions/1584062/how-to-wait-for-thread-to-finish-with-net — Theodoros Chatzigiannakis, Jan 15 '13 at 14:46
If you're wanting to create multiple threads, but you only want to run one at a time, you've picked the wrong tool. Threads are for when you *do* want parallel execution (to be possible - not guaranteed) — Damien_The_Unbeliever, Jan 15 '13 at 14:49
I would just use C# `async` instead of spawning a thread manually. Also if you are only going to process 1 at a time, why not just reuse the same thread? Creating a thread is a relatively intense process, and consumes something like 1MB of RAM. I don't see the point in throwing it away and creating another for the next file. If you are just trying to keep a UI responsive or something, use 1 thread and process the files in a normal loop, or use `async`. — CodingWithSpike, Jan 15 '13 at 14:54
Just start one thread and hand it a collection of CustomerFile. — Hans Passant, Jan 15 '13 at 15:01
@CodingWithSpike The `async` functionality in C# has nothing to do with spawning threads; it has everything to do with performing continuations on asynchronous tasks when they complete, but for it to do anything you still need someone, somewhere, to actually start another thread or other type of asynchronous operation. — Servy, Jan 15 '13 at 15:16
@Servy My point was more that `async` can handle keeping your UI responsive. That is a large part of why it exists (if your UI thread locks up in Win RT, your process is killed off, and everyone uses `async` to prevent it.) and spawning a thread may be overkill. See also Eric Lippert's answer here: http://stackoverflow.com/questions/8854269/c-sharp-5-async-await-thread-mechanics-feel-wrong — CodingWithSpike, Jan 15 '13 at 15:54
@CodingWithSpike Any `aysnc` method doesn't *always* need to have code running in another thread. In the case of IO, such as file or network IO there need be no other thread, but in this case he needs to do *work* in the background, which means a background thread, and `async` won't create that background thread, he needs to (or use the thread pool) which would most likely be done through `Task.Run`. — Servy, Jan 15 '13 at 15:59

Sten Petrov · Accepted Answer · 2013-01-15T15:01:18.363

2

How about add the files to a queue and process the queue on a another thread?

Queue<CustomerFile> files = new Queue<CustomerFile>()
foreach (CustomerFile f in CF)
  files.Enqueue(f);

BackgroundWorker bwk = new BackgroundWorker();
bwk.DoWork+=()=>{
    //Process the queue here
    // if you update the UI don't forget to call that on the UI thread
};

bwk.RunWorkerAsync();

edited Jan 15 '13 at 15:01

answered Jan 15 '13 at 14:47

Sten Petrov

10,943
1
41
61

`Queue` isn't thread save; so you will need to synchronize access. – Servy Jan 15 '13 at 14:51
@Servy - if the original thread just populates the queue (as shown here), and then the background worker is the only thread dequeueing (and is started after queueing is complete), then there's no need to synchronize. – Damien_The_Unbeliever Jan 15 '13 at 14:57

score 1 · Answer 2 · answered Jan 15 '13 at 14:50

This is the producer consumer model, and it's a very common requirement. In C# the BlockingCollection is ideal for this task. Have the producer add items to that collection, and then have the background task (you can have any number of them) taking items from that collection.

vlad · Answer 3 · 2013-01-15T14:58:01.267

it sounds like you could get away with just one background thread, that processes the queue. Something like this:

var q = new Queue();
foreach (var file in Directory.GetFiles("path"))
{
    q.Enqueue(file);
}

var t = new Task(() =>
    {
        while (q.Count > 0)
        {
            ProcessFile(q.Dequeue());
        }
    });
t.Start();

note that this is only appropriate if you don't have to modify the queue while the background thread is processing it. If you do, Servy's answer is correct: this is a pretty standard producer-consumer problem, with only one consumer. For more information on solving the producer/consumer problem, see Albahari's Threading in C#.

+1. There is no reason to create more than 1 thread in this case. — CodingWithSpike, Jan 15 '13 at 14:55

score 0 · Answer 4 · answered Jan 15 '13 at 17:27

the only thing you must do is put your loop in a thread for example something like this:

new Thread(()=>{
foreach (CustomerFile f in CF)
{
    btnGo.Enabled = false;
    UpdateProgressDelegate showProgress = new UpdateProgressDelegate(UpdateProgress);
    ProcessFile pf = new ProcessFile(this, showProgress, f._FileName, txtDestFolder.Text);
    pf.DoWork();

}
}).Start();

Loop files in a folder and add each to a thread, but only run one of the threads at a time

4 Answers4