1

I am writing concurrent software and after all docs I've red I was sure that multithreading module (same as asyncio) is binded to single python process and single core where it is run.

But recently my colleague said that in some source (he can't find exact link) he found that operation system will automatically distribute threads spawned by python between available cores. I've heard that operating system may manage threads but can not find explicit information about python threads behavior and now I have doubts.

So, how does operating system control python threads?

For example, https://realpython.com/async-io-python/, or https://medium.com/contentsquare-engineering-blog/multithreading-vs-multiprocessing-in-python-ece023ad55a points that for multicore usage multiprocessing module is required.

Sorry, I have no snippets clearly describing problem

Mars
  • 2,505
  • 17
  • 26
Vladimir Kolenov
  • 438
  • 1
  • 4
  • 14
  • 2
    Yes it probably will, but 2 *Python* threads won't run concurrently because of *GIL* (https://wiki.python.org/moin/GlobalInterpreterLock). – CristiFati Jun 14 '19 at 06:36
  • @CristiFati so saying roughly such usage of multiple cores is 'useless' in context of running python app and for CPU-bound tasks at all? – Vladimir Kolenov Jun 14 '19 at 06:40
  • @CristiFati could you please write you response as answer so I can mark it as solution? – Vladimir Kolenov Jun 14 '19 at 06:45
  • 1
    I'm saying that when it comes to parallel computing (*CPU* intensive operations that could be performed in parallel and could benefit from multiple cores), *threading* is to be avoided (*multiprocessing* is preferred). – CristiFati Jun 14 '19 at 06:47
  • @CristiFati I wonder... maybe the OS is smart enough to realize that it's essentially single-threaded and keep it on the same core to reduce inter-core overhead? – Mars Jun 14 '19 at 08:57
  • @Mars: the question is "a bit" to deep for my knowledge. But I imagine that once the *Python* process (with whatever of its threads being executed at a point), was flushed out by the scheduler from the *CPU* to make room for another process (with one of its threads), it doesn't matter if the *Python* process will be brought back to the same core, as the cache miss already happened when it was replaced. – CristiFati Jun 14 '19 at 09:34

1 Answers1

1

With the default python implementation, the answer is no. Multiple threads in python don't actually execute simultaneously, but instead execute instructions in turn, due to the Global Interpreter Lock. According to the Python Wiki, the GIL is used because python's memory management isn't thread safe, meaning that problems could result if two operations ran at the exact same time.

Some custom implementations (such as Jython and IronPython), don't use the GIL, and therefore should be able to take advantage of multi-core processors, however implementations with the GIL are stuck on using a single-core.

CrazySqueak
  • 584
  • 4
  • 8