Multiprocessing vs multithreading

Question

I have a GUI and, by the click of a button I have to start a big calculation.

I am using QtDesigner with python 2.7

At the beginning, I had the problem that my GUI became unresponsive while the big calculation was working.

I solved this using the Threading module:

class myMainWindow(QtGui.QMainWindow):

    def __init__(self, parent=None):
        #rest of the code here

class heavyCalculations(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)
        #rest of the code here

My doubt now is: Let's say that I want to start 4 heavy calculations, and I want to use all 4 cores of my computer working on time. For that I will need multiprocessing, but what should I do to avoid my GUI to become unresponsive, but at the same time to use 4 cores on time?

As far as I know, If I start a new process for each heavy calculation it works in parallel indeed, but my GUI becomes unresponsive.

And if I start 4 threads, it also works and the GUI is not affected, but this is not real parallel.

Any help?

Threads are not parallel, the processor jumps very fast from one to another, pretending to be "parallel" but it is not parallel if we call parallel when 4 processes work on 4 cores on time. It is not the same. — codeKiller, Oct 18 '14 at 20:58
I can get all cores to 100% with multithreading, so I would argue that one process != one core — Zaffy, Oct 18 '14 at 21:04
Due to its [GIL](http://www.jeffknupp.com/blog/2012/03/31/pythons-hardest-problem/), Python actually does not support native multithreading, so, in order to use multiple cores, multiple *processes* will be needed. — JimmyB, Oct 18 '14 at 21:12
Im afraid it does not work like that friend¡¡, If the load is very high, of course more cores work even with threads, but if you want to load all of them even with low load, you go for multiprocessing, actually the performance is better with multiprocessing...but anyway, let's wait for more answers, I may be wrong....I have not been working too much with threads. — codeKiller, Oct 18 '14 at 21:12
@Hanno I agree, but then it comes my question in the post...how to make my GUI responsive?? I have not been able to achieve this with my processes...only with threads... — codeKiller, Oct 18 '14 at 21:15
You should be able to start several Python threads each of which can then start and manage a single sub-process, mostly waiting for it to finish. — JimmyB, Oct 18 '14 at 21:19

score 1 · Answer 1 · edited May 23 '17 at 10:25

1

If you don't want your calculation processes to slow down your GUI process, you should make sure your GUI process runs at a higher priority than your calculation process. That way, whenever there is contention for a core (i.e. when there are more processes simultaneously wanting to run than your computer has CPU cores to run them on), the GUI process will get first dibs and the calculation process will have to wait until the GUI has finished doing whatever it wanted to do.

The easiest way to achieve that is to have your calculation processes lower their own priority as the first thing they do, before they start their calculations. As for how to get a Python process to lower its priority, see here.

edited May 23 '17 at 10:25

Community

1
1

answered Oct 19 '14 at 03:52

Jeremy Friesner

70,199
15
131
234

Thanks, that sounds great, I will need to test it. Just one more thing...Do I still need to run threads and processes or just processes?? I mean, if the sequence should be: Option 1: MainWindow----->runs a thread for each heavy calculation---> each thread runs a process. Or the Option 2: MainWindow--->runs a process for each heavy calculation. Thanks. – codeKiller Oct 19 '14 at 05:23
Just processes should be sufficient. I'm not sure what advantage having a thread to monitor each sub-process would add; I've never found it necessary. – Jeremy Friesner Oct 19 '14 at 14:56

Multiprocessing vs multithreading

1 Answers1