2

I have written an application for my company that essentially sends records from a text or CSV file in arrays of 100 records to a web service, to which it then returns the response, also in arrays of 100 records and writes them to another file. Currently, it is single threaded (processing sequentially using the Backgroundworker in Windows Pro Forms), but I am looking to multi-thread it utilizing a threadpool and concurrentqueue.

#region NonDataCollectionService
if (!userService.needsAllRecords)
{
    ConcurrentQueue<Record[]> outputQueue = new ConcurrentQueue<Record[]>();
    while ((!inputFile.checkForEnd()) && (!backgroundWorker1.CancellationPending)) 
    {
        //Get array of (typically) 100 records from input file
        Record[] serviceInputRecord = inputFile.getRecords(userService.maxRecsPerRequest);
        //Queue array to be processed by threadpool, send in output concurrentqueue to be filled by threads
        ThreadPool.QueueUserWorkItem(new WaitCallback(sendToService), new object[]{serviceInputRecord, outputQueue});
    }
}
#endregion



void sendToService(Object stateInfo)
{
    //The following block is in progress, I basically need to create copies of my class that calls the service for each thread
    IWS threadService = Activator.CreateInstance(userService.GetType()) as IWS;
    threadService.errorStatus = userService.errorStatus;
    threadService.inputColumns = userService.inputColumns;
    threadService.outputColumns = userService.outputColumns;
    threadService.serviceOptions = userService.serviceOptions;
    threadService.userLicense = userService.userLicense;

    object[] objectArray = stateInfo as object[];

    //Send input records to service
    threadService.sendToService((Record[])objectArray[0]);
    //The line below returns correctly
    Record[] test123 = threadService.outputRecords;
    ConcurrentQueue<Record[]> threadQueue = objectArray[1] as ConcurrentQueue<Record[]>;
    //threadQueue has records here
    threadQueue.Enqueue(test123);
}

However, when I check "outputQueue" in the top block, it is empty, even when threadQueue has records queued. Is this because I'm passing by value instead of passing by reference? If so, how would I do that syntactically with Threadpool.QueueUserWorkItem?

M. Vahdat
  • 29
  • 1
  • 1
  • 2
    This isn't related to passing by value or reference. Reviewing the code I see nothing glaringly wrong. However, I don't see anything waiting for your `ThreadPool` threads to complete. Also, you may be over-saturating the `ThreadPool`, potentially. Might want to consider using `Task` here along with `Task.WaitAll`. By default a `Task` will also utilize the `ThreadPool` but makes it much easier to manage concurrency. – Zer0 Jun 25 '18 at 21:15
  • You are using too low-level tools. A [`TransformBlock`](https://learn.microsoft.com/en-us/dotnet/api/system.threading.tasks.dataflow.transformblock-2) would probably serve you well here. Take a look at [this](https://stackoverflow.com/questions/50142516/how-to-sort-a-concurrentbag "How to sort a ConcurrentBag?") question. – Theodor Zoulias Feb 03 '23 at 03:54

0 Answers0