-1

I have been having this weird issue where my new laptop handles an albeit computationally intensive program I wrote worse than my last one. The code is a python program I wrote that measures errors in a numerical model for a large set of data. It uses the threading library to try and speed some parallelizable tasks up. The older laptop has an i7-8750h and 16gb ddr4 ram. The newer laptop has a i9-12900h and 40gb ddr5 ram (16 gb running quad channel, 24 gb running dual channel). In theory the this new laptop should be able to outperform the old one by a pretty significant margin, however it struggles significantly more with the program than the old one. I was wonder if anyone had any ideas about why this could be. My only thought is that the threading library interacts weirdly with the new intel 12th gen performance core and efficiency core setup.

I have tried restarting my newer laptop, and it has been handling other tasks very well, it seems to only have issues with this. I have also downloaded cpu-z and the benchmarks seem to be around normal.

Edit: The program usually took ~90 min to run on my last laptop, but there was randomity involved so it was inconsistent between runs. On the new one it gets through about 3% of the program in 90 minutes.

Ill try and upload an MRE soon.

double-beep
  • 5,031
  • 17
  • 33
  • 41
redhorn
  • 3
  • 2
  • Could it be that you have an 32-bit version installed of python? – Elias Kleppinger Jan 31 '23 at 07:50
  • Do you use any kind of IDE? Or you run Python program from terminal? – Hoang Minh Quang FX15045 Jan 31 '23 at 07:59
  • The program claims that It is 64 bit, python 3.11.1. It does say AMD64 when I check the version, but as I understand it that is how it should be even though I have an intel chip. – redhorn Jan 31 '23 at 08:01
  • I run it though an ide, I have used vs code on both laptops with the requisite extensions. – redhorn Jan 31 '23 at 08:01
  • [\[SO\]: Welcome to Stack Overflow](https://stackoverflow.com/tour). Check [\[SO\]: How do I ask a good question?](https://stackoverflow.com/help/how-to-ask) or **[\[SO\]: How to create a Minimal, Reproducible Example (reprex (mcve))](https://stackoverflow.com/help/minimal-reproducible-example)** for more asking related details. Also, [\[JonSkeet.CodeBlog\]: WRITING THE PERFECT QUESTION](https://codeblog.jonskeet.uk/2010/08/29/writing-the-perfect-question) might be a good point to start. – CristiFati Jan 31 '23 at 08:09
  • Add some details (clear measurement data). Also if you're using *threading* you're heading for big disappointment. https://wiki.python.org/moin/GlobalInterpreterLock, https://stackoverflow.com/q/52507601/4788546, ... – CristiFati Jan 31 '23 at 08:11
  • I'd add some more numerical data, but part of the issue is that on the new laptop it runs slower by orders of magnitude. Id say that the program would usually take about 90 minutes to run on my old laptop whereas it only gets though like 3% on my new one in that time, and I havent had the time to just let it run and see if it hangs. Ill try and see if I can come up with an MRE tomorrow, but I cant post the data that that is associated with this program. Are there any other better multithreading libraries for python? – redhorn Jan 31 '23 at 08:17
  • Please provide enough code so others can better understand or reproduce the problem. – Community Jan 31 '23 at 08:54

2 Answers2

0

You have not share the code so cannot pinpoint the issue, but here is the general explanation

Python multithreading does not guaranty parallelism due to Global Interpreter Lock limitation so at any given time only single thread is running inside the Python process.

Implementing multi threading for CPU bound operations can indeed sometime add more overhead. Hence you must use multiprocessing to achieve parallelism in true sense and improve the performance of your code. In multiprocessing each process has its own Python interpreter so they all can execute in parallel.

Multithreading only makes sense for IO bound operations.

user3844097
  • 66
  • 3
  • Ill definitely try and implement it with multiprocessing instead. I appreciate the response and explanation. Ill also try and upload an MRE soon. I still find it strange that 2 identical programs would run orders of magnitude slower on a more powerful pc. – redhorn Jan 31 '23 at 08:37
0

I was able to finally able to track down the issue. The older laptop has an older version of python (3.9.2) and on the newer one I installed the latest version (3.11.1). For some reason, with the program making use of the threading library it was running slower on the newer version of python. I installed 3.9.2 on the newer laptop and tried it with that interpreter version and it ran at speeds the same or greater than the older laptop. I have no idea why this is but for now I wont be using 3.11.1 I guess. I'd be interested to know exactly why the older version of python was running it faster.

redhorn
  • 3
  • 2
  • Multithreading in python has issues due to the inherent GIL on which python was built on. There are many threads related to this, and mostly a version of python has nothing to do with this. – Mr. Hobo Feb 03 '23 at 16:19
  • Yeah, the question though is not whether threading is a good or bad library, it's why an identical program on 2 pcs using it runs differently. And for whatever reason when I switch versions it affects performance. – redhorn Feb 04 '23 at 17:10