0

In this link is an example of finding prime numbers posted here. In his code, he compared sequential, threading, thread executor and process executor to find prime numbers. In his code, he found that ProcessPoolExecutor is the fastest:

Sequential Prime Finder Starting
9.708213827005238 seconds
Threading Prime Finder Starting
9.81836523200036 seconds
Processing Prime Finder Starting
3.2467174359990167 seconds
Thread Executor Prime Finder Starting
10.228896902000997 seconds
Process Executor Finder Starting
2.656402041000547 seconds

Question: Is it a good idea to have Process Executor (ProcessPoolExecutor) to manage incoming requests? I see that using ProcessPoolExecutor based on the example above could lead to low processing time but might decrease application throughput as I am not sure of the consequences of choosing to process requests using ProcessPoolExecutor versus Threading for example.

Avv
  • 429
  • 4
  • 17
  • 1
    neither really, requests are better handled with asyncio. – Ahmed AEK Nov 08 '22 at 15:40
  • @AhmedAEK. Thank you. In the post, it's mentioned that asyncio is better for IO bound tasks and they did not mention handling requests. Can you please elaborate more? – Avv Nov 08 '22 at 16:34
  • 1
    IO refers to all networking activities as well as a few pipes like stdin, so requests fall under asyncio too, although you'll have to use a different library for it that supports asyncio (aiohttp for example) – Ahmed AEK Nov 08 '22 at 16:37
  • @AhmedAEK. I am using PyZMQ (sub-pub) for network requests. So I meant by requests to handle the request after we got it on the server. – Avv Nov 08 '22 at 16:49
  • 1
    PyZMQ documentation refers to asyncio frequently, because its creators want you to use asyncio beause they know it will be better ... so you should be using it along with their documentation of how to use them together. – Ahmed AEK Nov 08 '22 at 17:02
  • @AhmedAEK. Thank you. I will read about `asyncio`. So you think it would be much faster than simply using threading/ProcessPool please? – Avv Nov 08 '22 at 17:04
  • 1
    i cannot answer the "much faster" part without seeing your entire codebase, but the "requests" part will have a higher throughput ... i can't say this about your entire code as it may be doing other things. – Ahmed AEK Nov 08 '22 at 17:07
  • @AhmedAEK. I read here that it's slower: "Is asyncio faster than threads? No. As Cal Peterson wrote: Sadly async is not go-faster-stripes for the Python interpreter." – Avv Nov 08 '22 at 17:07
  • 1
    it's best to always benchmark the code to see what is really faster for a specific code, taking my word or the library creators word for it is only good if you don't have the time to test things yourself and want a quick answer, if you have the time then nothing will give you a better answer than benchmarking your own code, keep in mind you have to benchmark the EXACT code that will work, the example you posted above has nothing to do with requests. – Ahmed AEK Nov 08 '22 at 17:11

0 Answers0