How To Speed up big List in C# with Parallel

Question

Hi i am Creating Big List in C# the Values are Calulated by local Variable and then i use them in list it Work but takes around 30 sec to complete just to create the final list i want to use Parallel as i see currently my program only use one thread

thnx any help Simplified Code Below

   decimal USDT  = Rs;
   decimal Pound = Rs;
   decimal Ruble= Rs;

 List<Author> authors = new List<Author>

//the Value are get by calling some other function 

//How To Speed up This part below

authors.Add(new Author { Name = "1", Price = value(1) , PriceinRs =USDT  * value(1)});
authors.Add(new Author { Name = "2", Price = Value(2) , PriceinRs =Ruble * value(2)});
authors.Add(new Author { Name = "3", Price = Value(3) , PriceinRs =Pound * value(3)});
...
//Total Around 2000 Other Authors

What is the `Value` and the `value`? Are they functions? Are they the same function misspelled? Could you include in the question at least the signature of these methods? Also do you really have 2,000 code lines starting with `authors.Add`, the one on top of the other, instead of doing a `for` loop in two lines of code? — Theodor Zoulias, Jul 27 '22 at 14:42
It would really be ideal to give us a [mcve]. Right now your question is too trivial. yes, `Parallel` can speed things up, but it can also make it slower. We need real code with real data to know for sure. — Enigmativity, Jul 28 '22 at 03:27

score 0 · Accepted Answer · answered Jul 27 '22 at 15:08

Here you have a full example with 4 ways but...

Be aware, parallelism is not always the better option especially with sets with lots of data and simple operations, like your example.

You can play with this example and change the length of data in Enumerable.Range(1, xxx) and see times results:

Example with 1000000*5 authors:

WAY 1, for 853
WAY 2, linq 448
WAY 3, parallel for each 1407
WAY 4, parallel with Partitioner 407

Example with 1000000*1 authors:

WAY 1, for 190
WAY 2, linq 90
WAY 3, parallel for each 209
WAY 4, parallel with Partitioner 304

Example with 1000000*10 authors:

WAY 1, for 962
WAY 2, linq 1747
WAY 3, parallel for each 1137
WAY 4, parallel with Partitioner 2690

In your case, i recommend you for or linq options.

CODE

// Set exchange rate
double usdt = 1.5667;
double btc = 21340.6532;

// Get authors with other funcion
var authors = Enumerable.Range(1, 1000000*10).Select(x=> new Author
{
    Name = "Author " + x.ToString(),
    usdtCoins = x*1000,
    btcCoins = x
}).ToList();


// WAY 1, for
var authorsCalculated = new List<Author>();
var timer = Stopwatch.StartNew();

foreach (var author in authors)
{
    authorsCalculated.Add(new Author
    {
        Name = author.Name,
        usdtCoins = author.usdtCoins,
        btcCoins = author.btcCoins,
        usdtMoney = author.usdtCoins * usdt,
        btcMoney = author.btcCoins * btc
    });
}

timer.Stop();
Console.WriteLine("WAY 1, for "+timer.ElapsedMilliseconds);

// WAY 2, linq
authorsCalculated = new List<Author>();
timer.Restart();

authorsCalculated = authors.Select(x=> new Author {
    Name = x.Name,
    usdtCoins = x.usdtCoins,
    btcCoins = x.btcCoins,
    usdtMoney = x.usdtCoins * usdt,
    btcMoney = x.btcCoins * btc
}).ToList();

timer.Stop();
Console.WriteLine("WAY 2, linq " + timer.ElapsedMilliseconds);

// WAY 3, parallel for each
authorsCalculated = new List<Author>();
var authorsCalculated_ConcurrentBag = new ConcurrentBag<Author>();
timer.Restart();

Parallel.ForEach(authors, author =>
{
    // Each parallel add one item on concurrent bag = safe list
    authorsCalculated_ConcurrentBag.Add(new Author
    {
        Name = author.Name,
        usdtCoins = author.usdtCoins,
        btcCoins = author.btcCoins,
        usdtMoney = author.usdtCoins * usdt,
        btcMoney = author.btcCoins * btc
    });
});

authorsCalculated = authorsCalculated_ConcurrentBag.ToList();

timer.Stop();
Console.WriteLine("WAY 3, parallel for each " + timer.ElapsedMilliseconds);


// WAY 4, parallel with Partitioner
// https://stackoverflow.com/questions/4031820/when-to-use-partitioner-class
authorsCalculated = new List<Author>();
var authorsCalculated_ConcurrentBag2 = new ConcurrentBag<List<Author>>();
timer.Restart();

Parallel.ForEach(Partitioner.Create(0, authors.Count()), range => {
    var innerList = new List<Author>();
    for (var index = range.Item1; index < range.Item2; index++)
    {
        innerList.Add(new Author
        {
            Name = authors[index].Name,
            usdtCoins = authors[index].usdtCoins,
            btcCoins = authors[index].btcCoins,
            usdtMoney = authors[index].usdtCoins * usdt,
            btcMoney = authors[index].btcCoins * btc
        });
    }
    authorsCalculated_ConcurrentBag2.Add(innerList);
});

foreach (var l in authorsCalculated_ConcurrentBag2)
{
    authorsCalculated.AddRange(l);
}

timer.Stop();

Console.WriteLine("WAY 4, parallel with Partitioner " + timer.ElapsedMilliseconds);

score -1 · Answer 2 · answered Jul 27 '22 at 11:08

-1

Would it not be faster to define PriceinRs in the Author class as PriceinRs => Price * Currency and define a Currency in the Author class, instead of assigning the value? Assuming USDT and Rubble and Pound are currencies.

answered Jul 27 '22 at 11:08

Mike Wijnen

154
2

thnx for replay but my main question is how can i use parallel processing in add list items to speed up overall process time – sanjay Jul 27 '22 at 11:18
To do this in parallel you can create for example 4 threads that create lists with 500 authors and when finished you can join them together, but I don't think it will be any faster. Probably slower. – Mike Wijnen Jul 27 '22 at 11:43

How To Speed up big List in C# with Parallel

2 Answers2

CODE