Threading out inside of loop to improve performance

Question

foreach (int tranQuote in transactionIds)
            {
                CompassIntegration compass = new CompassIntegration();
                Chatham.Business.Objects.Transaction tran = compass.GetTransaction(tranQuote);

                // then we want to send each trade through the PandaIntegration
                // class with either buildSchedule, fillRates, calcPayments or
                // a subset of the three
                PandaIntegrationOperationsWrapper wrapper = new PandaIntegrationOperationsWrapper { buildSchedule = false, calcPayments = true, fillRates = true };
                new PandaIntegration().RecalculateSchedule(tran, wrapper);

                // then we call to save the transaction through the BO
                compass.SaveTransaction(tran);
            }

Two lines here are taking a very long time. There's about 18k records in transactionIds that I do this for.

The GetTransaction and SaveTransaction are the two lines that take the most time, but I'd honestly just like to thread out what happens inside the loop to improve performance.

What's the best way to thread this out without running into any issues with the CPU or anything like that? I'm not really sure how many threads are safe or how to thread manage or stuff like that.

Thanks guys.

score 3 · Accepted Answer · answered Jan 19 '12 at 15:58

3

The TPL will provide the necessary throttling and managing.

//foreach (int tranQuote in transactionIds) { ... }
Parallel.ForEach(transactionIds, tranQuote => { ... } );

It does require Fx4 or later, and all the code inside the loop has to be thread-safe.
It's not clear if your GetTransaction and SaveTransaction are safe to be called concurrently.

answered Jan 19 '12 at 15:58

H H

263,252
30
330
514

Say they weren't, what would I have to do? Thanks Henk. – slandau Jan 19 '12 at 16:03
You would have to make them thread-safe, could be complicated. But they seem to be per-instance, and you have a new compass instance per thread. So I think you're OK. – H H Jan 19 '12 at 16:05
Hmm, okay I didn't know if maybe .NET had some built in way of queuing things up properly or something but running what it could in the thread before it had to pause or w/e. – slandau Jan 19 '12 at 16:07

score 3 · Answer 2 · answered Jan 19 '12 at 16:02

What do GetTransaction and SaveTransaction actually do that makes them slow? If it's work on the CPU then threading could certainly help. If the slowness comes from making database queries or doing file I/O, threading isn't going to do anything to make your disk or database faster. It might actually slow things down because now you have multiple simultaneous requests going to a resource that is already constrained.

score 1 · Answer 3 · answered Jan 19 '12 at 15:57

1

You can use Parallel foreach if order doesn't matter to improve the overall performance

answered Jan 19 '12 at 15:57

Haris Hasan

29,856
10
92
122

Yeah order doesn't matter, but is there a way I can specify the amount of threads and is there a safe thread limit or something? – slandau Jan 19 '12 at 15:58
Yes you can specify the amount of threads. Which part you want to make thread safe? – Haris Hasan Jan 19 '12 at 15:59
@slandau - the API will make smart choices about # of threads for you based on the #of CPU cores, etc. you *can* override it but that's rarely a good idea – Robert Levy Jan 19 '12 at 16:00
@RobertLevy - ah okay that was my main question. Did not know that. Thanks. – slandau Jan 19 '12 at 16:00

Threading out inside of loop to improve performance

3 Answers3