How can I convert this foreach code to Parallel.ForEach?

Question

I am a bit of confused about Parallel.ForEach.
What is Parallel.ForEach and what does it exactly do?
Please don't reference any MSDN link.

Here's a simple example :

string[] lines = File.ReadAllLines(txtProxyListPath.Text);
List<string> list_lines = new List<string>(lines);

foreach (string line in list_lines)
{
    //My Stuff
}

How can I rewrite this example with Parallel.ForEach?

Answer 1

Foreach loop:

• Iterations takes place sequentially, one by one
• foreach loop is run from a single Thread.
• foreach loop is defined in every framework of .NET
• Execution of slow processes can be slower, as they're run serially
  • Process 2 can't start until 1 is done. Process 3 can't start until 2 & 1 are done...
• Execution of quick processes can be faster, as there is no threading overhead

Parallel.ForEach:

• Execution takes place in parallel way.
• Parallel.ForEach uses multiple Threads.
• Parallel.ForEach is defined in .Net 4.0 and above frameworks.
• Execution of slow processes can be faster, as they can be run in parallel
  • Processes 1, 2, & 3 may run concurrently (see reused threads in example, below)
• Execution of quick processes can be slower, because of additional threading overhead

The following example clearly demonstrates the difference between traditional foreach loop and

Parallel.ForEach() Example

using System;
using System.Diagnostics;
using System.Threading;
using System.Threading.Tasks;
namespace ParallelForEachExample
{
    class Program
    {
        static void Main()
        {
            string[] colors = {
                                  "1. Red",
                                  "2. Green",
                                  "3. Blue",
                                  "4. Yellow",
                                  "5. White",
                                  "6. Black",
                                  "7. Violet",
                                  "8. Brown",
                                  "9. Orange",
                                  "10. Pink"
                              };
            Console.WriteLine("Traditional foreach loop\n");
            //start the stopwatch for "for" loop
            var sw = Stopwatch.StartNew();
            foreach (string color in colors)
            {
                Console.WriteLine("{0}, Thread Id= {1}", color, Thread.CurrentThread.ManagedThreadId);
                Thread.Sleep(10);
            }
            Console.WriteLine("foreach loop execution time = {0} seconds\n", sw.Elapsed.TotalSeconds);
            Console.WriteLine("Using Parallel.ForEach");
            //start the stopwatch for "Parallel.ForEach"
             sw = Stopwatch.StartNew();
            Parallel.ForEach(colors, color =>
            {
                Console.WriteLine("{0}, Thread Id= {1}", color, Thread.CurrentThread.ManagedThreadId);
                Thread.Sleep(10);
            }
            );
            Console.WriteLine("Parallel.ForEach() execution time = {0} seconds", sw.Elapsed.TotalSeconds);
            Console.Read();
        }
    }
}

Output

Traditional foreach loop
1. Red, Thread Id= 10
2. Green, Thread Id= 10
3. Blue, Thread Id= 10
4. Yellow, Thread Id= 10
5. White, Thread Id= 10
6. Black, Thread Id= 10
7. Violet, Thread Id= 10
8. Brown, Thread Id= 10
9. Orange, Thread Id= 10
10. Pink, Thread Id= 10
foreach loop execution time = 0.1054376 seconds

Using Parallel.ForEach example

1. Red, Thread Id= 10
3. Blue, Thread Id= 11
4. Yellow, Thread Id= 11
2. Green, Thread Id= 10
5. White, Thread Id= 12
7. Violet, Thread Id= 14
9. Orange, Thread Id= 13
6. Black, Thread Id= 11
8. Brown, Thread Id= 10
10. Pink, Thread Id= 12
Parallel.ForEach() execution time = 0.055976 seconds

Answer 2

string[] lines = File.ReadAllLines(txtProxyListPath.Text);
List<string> list_lines = new List<string>(lines);
Parallel.ForEach(list_lines, line =>
{
    //Your stuff
});

Answer 3

string[] lines = File.ReadAllLines(txtProxyListPath.Text);

// No need for the list
// List<string> list_lines = new List<string>(lines); 

Parallel.ForEach(lines, line =>
{
    //My Stuff
});

This will cause the lines to be parsed in parallel, within the loop. If you want a more detailed, less "reference oriented" introduction to the Parallel class, I wrote a series on the TPL which includes a section on Parallel.ForEach.

Answer 4

For big file use the following code (you are less memory hungry)

Parallel.ForEach(File.ReadLines(txtProxyListPath.Text), line => {
    //Your stuff
});

Answer 5

These lines Worked for me.

string[] lines = File.ReadAllLines(txtProxyListPath.Text);
var options = new ParallelOptions { MaxDegreeOfParallelism = Environment.ProcessorCount * 10 };
Parallel.ForEach(lines , options, (item) =>
{
 //My Stuff
});

Answer 6

I would like to add about parallel options. If you don't mentioned it, by default all RAM will be utilize for this which may give you problem in production. So better to add max degree of parallelism too in code.

Parallel.ForEach(list_lines, new ParallelOptions { MaxDegreeOfParallelism = 2 }, line =>
{
    
});