Throttle JavaScript function calls, but with queuing (don't discard calls)

Question

How can a function rate-limit its calls? The calls should not be discarded if too frequent, but rather be queued up and spaced out in time, X milliseconds apart. I've looked at throttle and debounce, but they discard calls instead of queuing them up to be run in the future.

Any better solution than a queue with a process() method set on an X millisecond interval? Are there such standard implementations in JS frameworks? I've looked at underscore.js so far - nothing.

Answer 1

Should be rather simple without a library:

var stack = [], 
    timer = null;

function process() {
    var item = stack.shift();
    // process
    if (stack.length === 0) {
        clearInterval(timer);
        timer = null;
    }
}

function queue(item) {
    stack.push(item);
    if (timer === null) {
        timer = setInterval(process, 500);
    }
}

http://jsfiddle.net/6TPed/4/

Answer 2

Here is an example which carries forward this (or lets you set a custom one)

function RateLimit(fn, delay, context) {
    var canInvoke = true,
        queue = [],
        timeout,
        limited = function () {
            queue.push({
                context: context || this,
                arguments: Array.prototype.slice.call(arguments)
            });
            if (canInvoke) {
                canInvoke = false;
                timeEnd();
            }
        };
    function run(context, args) {
        fn.apply(context, args);
    }
    function timeEnd() {
        var e;
        if (queue.length) {
            e = queue.splice(0, 1)[0];
            run(e.context, e.arguments);
            timeout = window.setTimeout(timeEnd, delay);
        } else
            canInvoke = true;
    }
    limited.reset = function () {
        window.clearTimeout(timeout);
        queue = [];
        canInvoke = true;
    };
    return limited;
}

Now

function foo(x) {
    console.log('hello world', x);
}
var bar = RateLimit(foo, 1e3);
bar(1); // logs: hello world 1
bar(2);
bar(3);
// undefined, bar is void
// ..
// logged: hello world 2
// ..
// logged: hello world 3

Answer 3

While the snippets offered by others do work (I've built a library based on them), for those who want to use well-supported modules, here are the top choices:

• the most popular rate limiter is limiter
• function-rate-limit has a simple API that works, and good usage stats on npmjs
• valvelet, a newer module, claims to do an even better job by supporting promises, but hasn't gained popularity yet

Answer 4

I needed a TypeScript version of this, so I took @Dan Dascelescu's fiddle and added types to it.

Please leave a comment if you can improve the typings

function rateLimit<T>(
  fn: (...args: Array<any>) => void,
  delay: number,
  context?: T
) {
  const queue: Array<{ context: T; arguments: Array<any> }> = []
  let timer: NodeJS.Timeout | null = null

  function processQueue() {
    const item = queue.shift()

    if (item) {
      fn.apply<T, Array<any>, void>(item.context, item.arguments)
    }

    if (queue.length === 0 && timer) {
      clearInterval(timer)
      timer = null
    }
  }

  return function limited(this: T, ...args: Array<any>) {
    queue.push({
      context: context || this,
      arguments: [...args],
    })

    if (!timer) {
      processQueue() // start immediately on the first invocation
      timer = setInterval(processQueue, delay)
    }
  }
}

Answer 5

I think this could get simpler and more reusable with decorators:

/**
 * Sleeps async for a given amount of time.
 * @param milisec
 * @returns
 */
function asyncDelay(milisec: number): Promise<void> {
  return new Promise<void>((resolve) => {
    setTimeout(() => { resolve(); }, milisec);
  });
}

/**
 * Generates a random int within the max and min range.
 * Maximum is exclusive and minimum is inclusive.
 * @param min
 * @param max
 */
export const getRandomInt = (
  min: number,
  max: number,
): number => (
  Math.floor(
    Math.random() * (
      Math.floor(max) - Math.ceil(min)
    ) + Math.ceil(min),
  )
);

/**
 * Will throttle a method by a fixed ms time, if number is passed.
 * if tuple, will throttle by a random time (ms) between each.
 * @param milliseconds
 * @returns
 */
function throttle(milliseconds: number | [number, number]): any {
  let lastCall = 0;
  return function (target: any, propertyKey: string, descriptor: PropertyDescriptor) {
    const originalMethod = descriptor.value;
    descriptor.value = async function (...args: any[]) {
      const now = Date.now();
      if (!lastCall) {
        lastCall = now;
        return originalMethod.apply(this, args);
      }
      const ms = Array.isArray(milliseconds)
        ? getRandomInt(milliseconds[0], milliseconds[1])
        : Math.round(milliseconds);

      const diff = now - lastCall;
      if (diff < ms) {
        await asyncDelay(ms - diff);
      }
      lastCall = Date.now();
      return originalMethod.apply(this, args);
    };

    return descriptor;
  };
}

The you can just do:

class MyClass {
  @throttle(1000)
  async fixedRateLimitedMethod() {
    await doSomething()
  }

  @throttle([1000, 5000])
  async randomRateLimitedMethod() {
    await doSomething()
  }
}

TSPlayground