13

I found this rate-limiting python decorator based on redis classes. How can I write a similar decorator that uses only what's available in the standard library that can be used as follows?

def ratelimit(limit, every):
    #  python magic 

@ratelimit(limit=1, every=2)
def printlimited(x):
    print x

# print one number every two seconds
for x in range(10):
    printlimited(x)

There are other answers on stackoverflow but they do not allow to specify the denominator.

Community
  • 1
  • 1
enrico.bacis
  • 30,497
  • 10
  • 86
  • 115

1 Answers1

20

You can use a threading.Semaphore to count and block the requests that are exceeding the limit, in combination with threading.Timer to schedule a function that releases the semaphore.

from threading import Semaphore, Timer
from functools import wraps

def ratelimit(limit, every):
    def limitdecorator(fn):
        semaphore = Semaphore(limit)
        @wraps(fn)
        def wrapper(*args, **kwargs):
            semaphore.acquire()
            try:
                return fn(*args, **kwargs)
            finally:                    # don't catch but ensure semaphore release
                timer = Timer(every, semaphore.release)
                timer.setDaemon(True)   # allows the timer to be canceled on exit
                timer.start()
        return wrapper
    return limitdecorator

I extended this idea and published a library on PyPI named limit.

enrico.bacis
  • 30,497
  • 10
  • 86
  • 115
  • 2
    This won't release the semaphore if the wrapped function throws an exception. You could use a `try..finally` clause to ensure that happens. – Paul M Furley Feb 02 '17 at 13:41