How to set maximum number of external processes the program can start at the same time?

Question

I need to run an external program for every PDF file in specified directory.

The problem is - how to limit the number of external program processes to user-specified value? I run it in the loop, like this:

foreach(string file in Directory.GetFiles(sourcePath))
{
    Process p = new Process();
    p.StartInfo.FileName = @"path\program.exe";
    p.StartInfo.Arguments = previouslySetArguments;
    p.Start();
}

Now the problem is that there is sometimes a really huge amount of files and with that code, all processes would be ran at the same time. It really slows the machine down.

Other idea is to put p.WaitForExit(); after the p.Start(); but then it would run only one process at a time, which on the other hand - slows down the whole work :)

What is the easiest way to limit processes number to run the exact amount of them at the same time? I want to let the user decide.

Let's say I want to run maximum 5 processes at once. So:
- first 5 processes starts in more-or-less the same time for first 5 files in the directory
- when one of them (doesn't matter, which) ends work, the next one starts - for the next file

possible duplicate of [Spawn processes, but only 5 at a time](http://stackoverflow.com/questions/4360720/spawn-processes-but-only-5-at-a-time) — Ben Voigt, Jun 18 '12 at 15:43
@Ramhound: TPL could be a nice, clean option but wouldn't be available if OP is using an older version of .Net. — Austin Salonen, Jun 18 '12 at 15:57

score 1 · Answer 1 · edited May 23 '17 at 12:10

1

If I were you I would look into a the producer-consumer model of queuing. It's intended to do pretty much exactly this, and there are lots of good examples that you can modify to suit your needs.

Here's an example: C# producer/consumer

And another example: http://msdn.microsoft.com/en-us/library/hh228601%28v=vs.110%29.aspx (this last one is for 4.5, but still valid IMO)

edited May 23 '17 at 12:10

Community

1
1

answered Jun 18 '12 at 15:57

Grant H.

3,689
2
35
53

score 0 · Answer 2 · edited May 23 '17 at 10:09

OK, so I tried the answers from this question, but - strange thing - I couldn't get them to work. I admit, I was in a hurry, I could made some mistakes...

For now, the quickiest and simpliest (and ugliest) method I've found is just a loop, like the code below. It works with the test program that just calls Thread.Sleep() with given command line argument as miliseconds.

Now, please, explain me, why is it not a good solution - I assume it is not the correct way (not only because the code is ugly), even if it works with this test example.

class Program
{
    // hardcoded, it's just a test:
    static int activeProcesses = 0;
    static int finishedProcesses = 0;
    static int maxProcesses = 5;
    static int runProcesses = 20;

    static string file = @"c:\tmp\dummyDelay.exe";

    static void Main(string[] args)
    {
        Random rnd = new Random();

        while (activeProcesses + finishedProcesses < runProcesses)
        {
            if (activeProcesses < maxProcesses)
            {
                Process p = new Process();
                p.EnableRaisingEvents = true;
                p.Exited += new EventHandler(pExited);
                p.StartInfo.WindowStyle = ProcessWindowStyle.Hidden;
                p.StartInfo.FileName = file;
                p.StartInfo.Arguments = rnd.Next(2000, 5000).ToString();
                p.Start();
                Console.WriteLine("Started: {0}", p.Id.ToString());
                activeProcesses++;
            }
        }

    }

    static void pExited(object sender, EventArgs e)
    {
        Console.WriteLine("Finished: {0}", ((Process)sender).Id.ToString());
        ((Process)sender).Dispose();
        activeProcesses--;
        finishedProcesses++;
    }
}

How to set maximum number of external processes the program can start at the same time?

2 Answers2