4

I have a python 3.8 script running multithreading with concurrent.futures module and works fine in MacOS Catalina (Intel). After migrated to MacOS Monterey (Apple Silicon). The python code runs for a long time due to using single thread. I'm using the python come with Anaconda which is x86_64 and running under Rosetta 2. Tried python 3.9 (from Anaconda) and got the same result. I will be appreciated if anyone can provide solution or workaround. Thanks.

Here is a testing code to show the problem. In old machine, it runs 2 rounds and completed in 10 seconds. In new machine, it runs 10 rounds and completed in 50 seconds.

import concurrent.futures
import time


pstart = time.time()
tasks = list(range(1,11))

def sleep_5s(task):
    time.sleep(5)
    print(f'Task {task} start at: {time.time()}')


def sleep_together(tasks):
    with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
        for i,task in zip(tasks, executor.map(sleep_5s, tasks)):
            pass


sleep_together(tasks)

print('Total run time', time.time()-pstart, 'seconds.')

Update:

I find the root cause. I didn't plug the power. The wifi will be disconnected after 5 mins when the display sleeps or screensaver starts. This is the new power tuning of MacOS. Here is the solution.

https://www.techrepublic.com/article/change-your-macos-power-settings-to-prevent-disconnecting-from-vpnwi-fi-when-the-computer-is-locked/

angle
  • 43
  • 4
  • When you say "Python 3.9" do you also mean via Rosetta or native Python? – riffle_shuffle Oct 29 '21 at 06:57
  • I use conda to create another environment with python 3.9 (Rosetta) to test whether it is related to python 3.8. I changed to a complete testing code so that you can test it if you have Apple Silicon/Monterey machine. Many thanks. – angle Oct 29 '21 at 08:00

1 Answers1

1

Your code runs as expected on macOS Big Sur, which is already Apple Silicon, with native Python 3.9.

It looks like the culprit would be Rosetta or Monterey and not your code, but I did see some weird bugs when working with Swift on Rosetta. Have you considered looking that way ?

Orijhins
  • 142
  • 5
  • I will test with native Python 3.9 tonight and update later. Many thanks. – angle Oct 29 '21 at 08:47
  • After further investigation and find the multithreading works fine but some of multithreading code was frozen. It seems the multithreading loop dead locked. I don't know it is related to the yfinance module (used to download stock info). But the problem is fixed after changed to native Python 3.9. Changed from Anaconda to minforge. Many thanks. – angle Oct 30 '21 at 13:33