2

I've been trying to synchronize python multiprocessing. So far I ran into these issues:

  1. multithreading can only use one core due to the GIL, however in other languages it is possible to make multiple threads use multiple cores inside a single process and have them access the same object

  2. multiprocessing can use multiple cores but the shared memory is extremely slow (200 - 1000x times slower) especially with complex object (class A object contains a list of class B objects and a bunch of int/float, each B object contains a list of class C objects etc.)

  3. Jython and ironpython do not have the GIL but making certain modules work with them is problematic (anything implemented in C / C++)

So is there a way to make multithreading use multiple cores in Python inside a single process? Not asking for a complete implementation, just if it's possible and if so how

martineau
  • 119,623
  • 25
  • 170
  • 301
qwerty_99
  • 640
  • 5
  • 20
  • 1
    The GIL is a big problem in Python, because so many of Python's internal data structures need to be coordinated across threads. Multithreading really can use multiple cores, but the GIL keeps them running only one at a time. – Mark Ransom Aug 25 '20 at 17:59
  • Answers to your question https://stackoverflow.com/questions/7542957/is-python-capable-of-running-on-multiple-cores – alex_noname Aug 25 '20 at 18:06
  • you've crashed into an unicorn. use multiple processes, a single process can't do it. – Pedro Rodrigues Aug 25 '20 at 19:20
  • thanks. could putting the stuff in C++ do it? like generating python object with C++ threads – qwerty_99 Aug 29 '20 at 17:08
  • Please check the answer by me: https://stackoverflow.com/questions/10568358/jython-multithreading – Om Prasad Nayak Dec 18 '20 at 07:10

0 Answers0