"Throttle" a function

Question

I'm developing an MMORPG game server, while this approach is not needed for 80% of the functions within the game server, the 20% of the functions that would use it account for 99% of resource usages.

I'm trying to accomplish a throttled function. Say you call a massive function, or you're searching a massive data structure for a specific criteria. That one call being fired could result in massive CPU usage.

For example, a massive function, when called, utilizes a steady 30% of the CPU over 4 seconds to complete, say we're searching millions of entries in an auction system by criteria. Imagine we have 4 people doing this at once. I want to make that same method to take only 1% CPU or less, taking perhaps more than 15-20 seconds to return the same result without starving hardware.

As of now I'm using this class.

public class AsyncLazy<T> : Lazy<Task<T>> 
{ 
    public AsyncLazy(Func<T> valueFactory) : 
        base(() => Task.Factory.StartNew(valueFactory)) { }
    public AsyncLazy(Func<Task<T>> taskFactory) : 
        base(() => Task.Factory.StartNew(() => taskFactory()).Unwrap()) { } 
}

This doesn't block in initialization of the value, and doesn't block when consuming the lazy value. However, during massive operations the executing the operation starves the hardware to complete the task, resulting in 30% CPU usage over 4 seconds. My goal is to throttle calls over a virtually unlimited amount of time so they can complete their task without starving hardware.

Edit: Thanks to Scott Chamberlin for correcting me with proper "lingo". I'm not looking to call a method lazily, but I'm looking to "Throttle" specific method calls.

Lazy has a specific meaning in C#, what you are looking for is Throttling, not Lazy. — Scott Chamberlain, Jun 30 '15 at 21:01
Thank you, I didn't know that. So my goal here is to throttle specific calls.. — Scotty, Jun 30 '15 at 21:04
Please update your question (and its title) to re-word it to focus on the fact you want to throttle the execution, not execute it lazily. — Scott Chamberlain, Jun 30 '15 at 21:05
@Scotty Even "throttling" is complex, because you can't really control the thread scheduler that Windows uses... What you can do is lower the priority of the thread... But it is "bad" to do with threads from the thread pool. See for example http://stackoverflow.com/q/1578599/613130 and http://stackoverflow.com/q/5589376/613130 — xanatos, Jun 30 '15 at 21:05
You can try creating threads yourself, and setting the priority on these threads to low. If you choose to change the priority of the threads in the thread pool, make sure to store their previous value and restore it in a `finally` block. — willaien, Jun 30 '15 at 21:07
Is there any way to do this via reflection? Basically generate code at runtime to throttle any method? — Scotty, Jun 30 '15 at 21:08
Reflection will not help you with this. Do you have control of the code you are wanting to throttle or will you need a solution that treats the code as a "Black Box" and only does throttling options from outside the method. — Scott Chamberlain, Jun 30 '15 at 21:11
BTW, "searching millions of entries in an auction system by criteria" should require 0% CPU by your code, that kind of action should be performed by your database. You should just send the query off then block or await the result. No CPU should be needed. — Scott Chamberlain, Jun 30 '15 at 21:13
In order to throttle I think you need to be able to break the `Func` down into multiple discrete tasks. Is that possible? — Enigmativity, Sep 26 '15 at 01:33
via another enumeration, where `MoveNext()` is called by a task manager. — Scotty, Sep 26 '15 at 19:38

DB Tech · Answer 1 · 2015-10-15T15:14:28.280

How can you throttle a function using the your programming language?

As mentioned by willaien, Thread Priority is one of the only CPU throttling mechanism you have inside the process, and even that is indirect. This is because it is not an app's responsibility to directly control CPU utilization; it is the OS and processor's responsibility to manage system resources such as threading. This allows the OS to prevent a single app from crashing everything.

How can you throttle your function outside of your programming language?

You can break up the intense calculation into a completely separate service. Then place this service on another machine that you can directly control the resources of from a system or OS level (for example adding more memory or CPU power by hardware or using VMs). With a VM, you can "throttle" the function by setting the VMs CPUs. You can "throttle" a pure hardware solution or VMs using middleware like a load balancer or using a queue to limit the number of in-process requests. This would satisfy your requirement of each calculation taking as long as it needs. And that other service could simply be a distributed cache, (as Scott Chamberlain mentioned) a database stored proc, or query where the database is on a different machine.

If you take this route, you may want to investigate .NET's async pattern. This would also ensure that your original service is not thread bound by threads that are blocking when using your distributed service.

how to throttle? -- simple: call `sleep(delay)`. Here's how to [draw a sine wave using CPU usage](http://stackoverflow.com/q/551494/4279). In practice, set cpu affinity, process niceness and let OS to schedule it. — jfs, Oct 15 '15 at 07:03
True. If you do that in a loop, you can "throttle" the function. But you should weigh in this: http://stackoverflow.com/questions/8815895/why-is-thread-sleep-so-harmful And consider how maintainable it is to precisely throttle using a sleep. — DB Tech, Oct 15 '15 at 15:11
I've mentioned `sleep()` to point out that thread priority is not "the only" solution (your edit acknowledges it) though I don't consider it practical and therefore there is the second sentence that starts with "in practice". — jfs, Oct 15 '15 at 15:32

"Throttle" a function

1 Answers1