How to use multiple core with a webservice based on Python Klein

Question

I am writing a web service based on Klein framework

https://klein.readthedocs.io/en/latest/index.html

At this stage I am stress testing my service, it can handles about 70 requests per second on amazon t2.medium instance. But when I use top to check the server, it only use 100% of CPU. I think amazon t2.medium instance should have 2 cpu, so I wonder is there a way to change in my web service code to use all of the possible cpus and hopefully handle more requests.

I've read python documentations and found the multiprocessing module but I am not sure will that be the right solution to it. Right now the main function of my web service is

APP = Klein()
if __name__ == "__main__":
    APP.run("0.0.0.0", SERVER_PORT)

Is there a straight forward fix to make this service being able to use multiple cpu to process the incoming requests? Thank you for reading the question.

score 2 · Accepted Answer · edited May 23 '17 at 12:32

It's certainly possible to use multiprocessing and it sure as heck is easy to spin up processes.

from multiprocessing import Process
from klein import Klein

def runserver(interface, port, logFile):
    app = Klein()
    @app.route('/')
    def heyEarth(request):
        return 'Hey Earth!'
    app.run(interface, port, logFile)

process_list = []
for x, port in enumerate([8000, 8001, 8002, 8003]):
    logfilename = open('localhost' + str(port) + '.log', 'a')
    process_list.append(Process(target=runserver, args=('localhost', port, logfilename)))
    process_list[x].daemon = True
    process_list[x].start()

process_list.pop().join()

In an enterprise environment it's better and more reliable to run behind a dedicated load balancer like nginx. So the snippet above should only be used to start the web servers, then all your load balancing should be handled by a dedicated load balancer.

Keep the multiprocess code to a bare minimum or else basic things like debugging and shared system files start to become an annoyance. And that's the "normal" stuff, there are TONS of ABNORMALITIES that can arise and no one will be able to help you because you don't know what's happening yourself. Just running this snippet, I noticed a few weird things with signals and Twisted. I think it could be fixed if I ran all klein imports in the runserver().

Get informed, learn from other's mistakes, heed the warnings of those who have been burned by multiprocessing and go make a kick ass app! Hope this helps :D

References

Tornado: Running and Deploying - it's a Tornado doc, but it should still help
Multicore TwistedWeb - This is the "Twisted" approach to multiprocesses by a core developer. Worth a read for anyone looking to run Twisted and spawn processes.

Are these extra processes can all being used to process the request targeting the same port? Or it always need to be use multiple port? — JLTChiu, Sep 08 '16 at 20:25
If it's on the same machine then it would need to be different ports (at least for my implementation) — notorious.no, Sep 08 '16 at 20:59
Tried this before, wonder is there a solution to this without the need of using multiple ports, otherwise need to change the api (which brings more problems) — JLTChiu, Sep 21 '16 at 14:11

How to use multiple core with a webservice based on Python Klein

1 Answers1

References