how do I add to a parallel list in a foreach loop

Question

I was reading this post Limit the number of parallel threads in C# and trying to use it to send multiple files via ftp at the same time:

Perhaps something along the lines of:

    ParallelOptions options = new ParallelOptions();

    options.MaxDegreeOfParallelism = 4;
    Then in your loop something like:

    Parallel.Invoke(options,
     () => new WebClient().Upload("http://www.linqpad.net", "lp.html"),
     () => new WebClient().Upload("http://www.jaoo.dk", "jaoo.html"));

I am trying to add the files in my directory to the Invoke but I wasn't sure how to add them:

  var dirlisting = Directory.GetFiles(zipdir, "*.*", SearchOption.TopDirectoryOnly);
  if (!dirlisting.Any())
  {
    Console.WriteLine("Error! No zipped files found!!");
    return;
  }

  foreach (var s in dirlisting)
  {
    var thread = new Thread(() => FtpFile.SendFile(s));
    thread.Start();
  }

I wasn't sure how to add them to the list of files to send. I only want 3 to go up at a time.

How do I add a thread for each file in the directory listing to be sent?

`dirlisting.AsParallel().WithDegreeOfParallelism(3).ForAll(FtpFile.SendFile)` , C# is pretty amazing at this sort of thing, you shouldn't worry about threads, although - I'd recommend looking into asynchronous I/O if you want to do it efficiently, namely - with the newer `HttpClient`. — Benjamin Gruenbaum, May 26 '14 at 21:32

score 0 · Answer 1 · answered May 26 '14 at 21:33

0

Something along these lines should do the trick

ParallelOptions o = new ParallelOptions();
o.MaxDegreeOfParallelism = 3;

Parallel.ForEach(dirlisting, o, (f) => 
{
   Ftp.SendFile(f);
});

answered May 26 '14 at 21:33

sondergard

3,184
1
16
25

score 0 · Answer 2 · edited May 23 '17 at 12:20

Parallel.For family of APIs is for CPU-bound tasks. You have IO-bound tasks here. Ideally, you should be using some asynchronous I/O Task-based API.

HttpClient doesn't support FTP uploads, but you can use FtpWebRequest.GetRequestStreamAsync and Stream.WriteAsync. You can use WebClient.UploadFileAsync too, but you'd need to create a new WebClient instance per each UploadFileAsync, as WebClient doesn't support multiple operation in parallel on the same instance.

Then you can use TPL Dataflow Library or just SemaphoreSlim to limit the level of parallelism. For examples, check "Throttling asynchronous tasks".

score 0 · Answer 3 · answered Aug 01 '19 at 08:04

0

These lines Worked for me.

var options = new ParallelOptions { MaxDegreeOfParallelism = Environment.ProcessorCount * 10 };
Parallel.ForEach(dirlisting, options, (item) =>
{
   FtpFile.SendFile(item)
});

answered Aug 01 '19 at 08:04

Prince Prasad

1,528
1
16
20

how do I add to a parallel list in a foreach loop

3 Answers3