Running maximum threads: Automatic performance adjustment

Question

I'm developing an app which one scans thousands copies of a struct; ~1 GB RAM. Speed is important.

     ParallelScan(_from, _to);  //In a new thread

I manually adjust the threads count:

     if (myStructs.Count == 0) { threads = 0; }
     else if (myStructs.Count < 1 * Number.Thousand) { threads = 1; }
     else if (myStructs.Count < 3 * Number.Thousand) { threads = 2; }
     else if (myStructs.Count < 5 * Number.Thousand) { threads = 4; }
     else if (myStructs.Count < 10 * Number.Thousand) { threads = 8; }
     else if (myStructs.Count < 20 * Number.Thousand) { threads = 12; }
     else if (myStructs.Count < 30 * Number.Thousand) { threads = 20; }
     else if (myStructs.Count < 50 * Number.Thousand) { threads = 30; }
     else threads = 40;

I just wrote it from scratch and I need to modify it for another CPU, etc. I think I could write a smarter code which one dynamically starts a new thread if CPU is available at the moment:

• If CPU is not %100 start N thread
• Measure CPU or thread process time & modify/estimate N
• Loop until scan all struct array

Is there anyone think that "I did something similar" or "I have a better idea" ?

UPDATE: The solution

    Parallel.For(0, myStructs.Count - 1, (x) =>
    {
         ParallelScan(x, x); // Will be ParallelScan(x);

    });

I did trim tons of code. Thanks people!

UPDATE 2: Results

Scan time for 10K templates

• 1 Thread: 500 ms
• 10 Threads: 300 ms
• 40 Threads: 600 ms
• Tasks: 100 ms

Answer 1

The standard answer: Use Tasks (TPL) , not Threads. Tasks require Fx4.

Your ParallelScan could just use Parallel.Foreach( ... ) or PLINQ (.AsParallel()).

The TPL framework includes a scheduler, and ForEach() uses a partitioner, to adapt to CPU cores and load. Your problem is most likely solved with the standard components but you can write custom-schedulers and -partitioners.

Answer 2

Actually, you won't get much benefit from spanning 50 threads, if you CPU only has two cores (even if each of them supports hyperthreading). If will actually run slower due to context switching which will occur every once in a while.

That means you should go for the Task Parallel Library (.NET 4), which takes care that all available cores are used efficiently.

Apart from that, improving the asymptotic duration of your search algorithm might prove more valuable for large quantities of data, regardless of the Moore's law.

[Edit]

If you are unable/unwilling to use .NET 4 TPL, you can start by getting the information about the current number of logical processors in the system (use Environment.ProcessorCount or check this answer for detailed info). Based on that number, you can partition your data and span a fixed number of threads. That is much simpler that checking the CPU utilization, and should prevent creating unnecessary threads which are starved anyway.

Answer 3

OK, sorry to keep going on but first to compile my comments:

• Unless you have a very, very, very, good reason to think that scanning these structs will take any more than a handful of microseconds and that really, really, really matters, it's not a good idea to do this kind of optimisation. If you really want to do it, you should have one thread per core. But really - don't. If it's just 50,000 structs and you're doing something simple with them, don't bother.
• FYI, starting a new thread takes a good amount of time (a measurable part of a second, several milliseconds).
• How long does this operation take? It's very unlikely that it's useful for you to optimize multithreading like this. It will give you the worst improvement. Better improvement will be gained by a better algorithm, or not having to depend on this weird invented multithreading scheme.

I'm confused about your performance fixation partly because you say you're looking through 50,000 structs (a very quick and easy operation) and partly because you're using structs. Without boxing that's a value type and if you're passing them around threads you're copying data rather than references, i.e. using more memory. My point being that that's a lot of data/memory, unless the structs are small, in which case, what kind of processing can you possibly be doing on them that takes so long as to think about 40+ threads in parallel?

If performance is truly incredibly important and your goal, and you're not simply trying to do this as a nice engineering exercise, please share information about what kind of processing you're doing.