PyQt MainWindow using multiprocessing on Windows

Question

I try to create a PyQt application. In order to run process in background and keep the PyQt5 application available for new instruction, I want to use multiprocessing. On the Windows OS, when I call a function from the Qt MainWindow class with multiprocessing.process, I have an error about pickling this class. But it is running find on Linux.

Here is an example:

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import sys          
import multiprocessing

# 
from PyQt5.QtWidgets import *
from PyQt5.QtGui import *
from PyQt5.QtCore import *


class Frame_main(QMainWindow):

    def __init__(self, parent = None):
        super(Frame_main, self).__init__(parent)
        self.thrd_list=[]
        self.initUI()

    def initUI(self):

        # Button 
        btn_run = QPushButton('Run', self)      
        btn_run.clicked.connect(lambda: self.ThdSomething(self.DoRun) ) #
        btn_stop = QPushButton('Stop', self)     
        btn_stop.clicked.connect(self.DoStop)    


        ### TEXT Edit       
        self.textEdit_term = QTextEdit("terminal: \n ")
        self.textEdit_term.append("")
        self.textEdit_term.setStyleSheet("color: rgb(255, 255, 255); background-color: black;")
        self.textEdit_term.setLineWrapMode(QTextEdit.NoWrap)
        self.textEdit_term.setToolTip(' terminal message ')
        self.textEdit_term.setStatusTip('textEdit1')

        ### LAYOUT
        Wid_main = QWidget()                    # 
        grid_major = QGridLayout()              # 
        grid_major.addWidget( btn_run, 1, 5)
        grid_major.addWidget( btn_stop, 2, 5)
        Wid_main.setLayout(grid_major)

        self.setCentralWidget(Wid_main)

        ### Affichage 
        self.show()                 #

    # 
    def DoRun(self):
        print('g starts')
        time_start=time.time()
        time_stop=time.time()
        name='bob'
        n=0
        while time_stop-time_start <2 :
            n+=1
            time_stop=time.time()
            time.sleep(0.8)
            print ('hola', name,n, flush=True)
        print('g stops')

    def DoStop(self):
        ''' subourtine kill all the thread '''
        print('stop action detected')
        while len(self.thrd_list) > 0 :
            print("Terminating the job: {}".format(self.thrd[-1].pid) )
            os.kill(self.thrd[-1].pid, signal.SIGTERM) 
            self.thrd_list[-1].terminate()

            self.thrd_list.pop()


    def ThdSomething(self, job):
        ''' subourtine to create a new thread to do the job subroutine '''
        arg=''
        p=multiprocessing.Process(target=job, args=(arg))
        self.thrd_list.append( p )
        p.start()
        print("Start the job GUI: {} with PID: {}".format(str(job) ,self.thrd[-1].pid), file=sys.stdout )


    def closeEvent(self, event):
        ''' subroutine to define what happen when closing the main frame'''
        self.statusBar().showMessage('waiting for a respond') 
        reply = QMessageBox.question(self, 'Message',
            "Are you sure to quit?", QMessageBox.Yes | 
            QMessageBox.No, QMessageBox.No)
        if reply == QMessageBox.Yes:
            event.accept()
        else:
            event.ignore()


# Main
if __name__ == '__main__':  
    # Create QApplication and Main Frame
    Qapp = QApplication(sys.argv)   # creation de lappli Qt
    Qapp.setStyle("fusion")         # 
    frame1 = Frame_main()           # 

    sys.exit(Qapp.exec_())          #

EDIT:

I found several similar case, like : Python: multiprocessing in pyqt application but none of them helped. I think this might be linked to fact that my case is using function and attributes of the MainWindow class.

@eyllanesc The first error is `TypeError: can't pickle Frame_main objects`, and the second on is `File "E:...\Python36\lib\multiprocessing\reduction.py", line 87, in steal_handle _winapi.DUPLICATE_SAME_ACCESS|_winapi.DUPLICATE_CLOSE_SOURCE) PermissionError: [WinError 5] Acces refused` — R. N, Jan 13 '17 at 09:54

score 4 · Accepted Answer · answered Jan 14 '17 at 05:46

You are correct that this is due to the attempt to fork on a method within the GUI class. Unfortunately windows doesn't really have the ability to fork processes the way linux does, which is why your code works on linux and not on windows. The Python 3 documentation for the multiprocessing library has some useful information on the differences and what the default method for starting a new process is under different platforms.

To answer your question: because your target is a method associated with a GUI object, it has to send the object state to the new process but fails to pickle it because a PyQt GUI object is far too complex to pickle.

If you rewrite your code so that the target function (and args specified) are picklable, your code will work.

PyQt MainWindow using multiprocessing on Windows

EDIT:

1 Answers1

Linked