0

Is there any way I can run a function in parallel with batches of a large dataset on multiple threads in c#?

So I have a list of data with approximate size of 32000 lines. I run the function below which reads each line of the dataset and verifies it. The idea is to separate the dataset into chunks of 5000 and concurrently apply the function below to each chunk/batch.

    private void AccountNumberCheck(List<Invoice> invoices, string VendorID)
    {
        try
        {
            using (var context = new ApplicationContext())
            {
                foreach (var invoice in invoices)
                {
                    var invoiceDB = context.Invoices.Find(invoice.Id);
                    var accountNumber = context.Accounts.Where(m => m.Account_Number == invoice.Account_Number && m.VendorID == VendorID);
                    if (accountNumber.Count() > 0)
                    {
                        var activeAccount = accountNumber.Any(m => m.Active_Status == false);
                        if (activeAccount == true)
                        {
                            invoiceDB.ExceptionFlag = true;
                            invoiceDB.ExceptionComments = invoiceDB.ExceptionComments + "The Account Number is Inactive.";
                        }
                        else
                        {
                            invoiceDB.ExceptionFlag = false;
                        }

                    }
                    else
                    {
                        invoiceDB.ExceptionFlag = true;
                        invoiceDB.ExceptionComments = invoiceDB.ExceptionComments + "The Account Number does not exist. ";
                    }

                    context.Entry(invoiceDB).State = EntityState.Modified;
                    context.SaveChanges();

                }
            }
        }
        catch (Exception ex)
        {
        }
    }
user2806570
  • 821
  • 2
  • 12
  • 25
  • Your main blocker here is that EF (I presume that's what your context is?) is not [thread safe](https://stackoverflow.com/questions/12827599/parallel-doesnt-work-with-entity-framework). That's going to make simplest solution here (`Parrellel.ForEach`) trickey. So I don't think there is an easy solution here. – Liam Jul 27 '18 at 10:03
  • Yes it is using EF. The application is built in .NET MVC 5. Any advice on what would be the easiest "complicated solution"? – user2806570 Jul 28 '18 at 17:49

0 Answers0