0

I'm trying to add multiprocessing to my tkinter app and I've been having issues with the error: TypeError: cannot pickle '_tkinter.tkapp' object. I had a look at the solution proposed in the question here and tried to implement my own version of it and this appears to have solved this particular error but now I instead I have constant OSError: [Errno 22] Invalid argument:

What I aspire to have the code do is that some calculation is being performed in the background and results of this calculation are being put into the Queue (here just integers but will be Numpy arrays in the actual code). The GUI application then displays some statistics and results to the user.

from multiprocessing import Process, Queue
from queue import Empty
import tkinter as tk
from tkinter import Tk

class FooUI(Process):
    def __init__(self, q: Queue):
        super().__init__(target=self, args=(q,))
        self.queue = q
        self.duh = []
        self.root = Tk()
        self._create_interface()
        self.root.after(100, self._check_queue)
        self.root.mainloop()
        
    def _check_queue(self):
        try:
            out = self.queue.get_nowait()
            if out:
                self.duh.append(out)
                print(self.duh)
                return
        except Empty:
            pass
        self.root.after(100, self._check_queue) 
    
    def _create_interface(self):
        self.root.geometry("100x100")
        b = tk.Button(self.root, text='Start', command=self.calc)
        b.grid(row=0, column=0)
    
    def calc(self):
        p = Process(target=do_calc)
        p.start()     
        
def do_calc(q: Queue):
    for i in range(20):
        q.put(i**2)


If __name__ == '__main__':
    q = Queue()
    f = FooUI(q)
    f.start()

And here is the traceback:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\cherp2\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\cherp2\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py", line 125, in _main
    prepare(preparation_data)
  File "C:\Users\cherp2\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\cherp2\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "C:\Users\cherp2\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 264, in run_path
    code, fname = _get_code_from_file(run_name, path_name)
  File "C:\Users\cherp2\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 234, in _get_code_from_file
    with io.open_code(decoded_path) as f:
OSError: [Errno 22] Invalid argument: 'C:\\python\\block_model_variable_imputer\\<input>'
Traceback (most recent call last):
  File "<input>", line 3, in <module>
  File "C:\Users\cherp2\AppData\Local\Programs\Python\Python38\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\cherp2\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\cherp2\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "C:\Users\cherp2\AppData\Local\Programs\Python\Python38\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\cherp2\AppData\Local\Programs\Python\Python38\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_tkinter.tkapp' object
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\cherp2\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\cherp2\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py", line 125, in _main
    prepare(preparation_data)
  File "C:\Users\cherp2\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\cherp2\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "C:\Users\cherp2\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 264, in run_path
    code, fname = _get_code_from_file(run_name, path_name)
  File "C:\Users\cherp2\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 234, in _get_code_from_file
    with io.open_code(decoded_path) as f:
OSError: [Errno 22] Invalid argument: 'C:\\python\\block_model_variable_imputer\\<input>'

I've been trying for a while to get it to work. Any help will be greatly appreciated!

NotAName
  • 3,821
  • 2
  • 29
  • 44
  • Why do you need multiprocessing for this? Why can't you just use threading? The code as you have it creates a second process where the UI will run, then a third process to do the calculation, while the main process apparently just exits. Did you try simply `FooUI(Queue())`? – Tim Roberts Feb 22 '21 at 04:05
  • @TimRoberts, I need multiprocessing to run multiple calculations in parallel but if I just try to spawn processes for calculations from the main process where tkinter instance is running it will result in this error: `TypeError: cannot pickle '_tkinter.tkapp' object`. This way I can avoid this TypeError but another one comes up. – NotAName Feb 22 '21 at 05:08
  • I highly doubt if `tkinter` support multiprocessing. Widgets created in one thread cannot communicate with the ones in another. – Delrius Euphoria Feb 22 '21 at 05:56
  • @CoolCloud, but what if in the other process we have not a widget but some calculation completely unrelated to tkinter? Will it still not work? – NotAName Feb 22 '21 at 05:57
  • Yes that should be fine. All tkinter stuff should be in a single process. – Delrius Euphoria Feb 22 '21 at 06:01
  • @CoolCloud so let me get this straight - I should not spawn calculation (`do_calc()`) from inside the process that has `tkinter` running? I should then send a message to the queue and have a completely unrelated process spawn the calculation? – NotAName Feb 22 '21 at 06:05
  • Well, I am not sure about that, guess it will be ok. – Delrius Euphoria Feb 22 '21 at 07:13

1 Answers1

1

You do the subclass of Process() in a wrong way. You need to override the run() method instead of passing target option.

from multiprocessing import Process, Queue
from queue import Empty
import tkinter as tk

class FooUI(Process):
    def __init__(self, q: Queue):
        super().__init__() # don't pass target and args options
        self.queue = q
        self.duh = []
    
    # override run() method and create the Tk() inside the function
    def run(self):
        self.root = tk.Tk()
        self._create_interface()
        self.root.after(100, self._check_queue)
        self.root.mainloop()
        
    def _check_queue(self):
        try:
            out = self.queue.get_nowait()
            if out:
                self.duh.append(out)
                print(self.duh)
                #return
        except Empty:
            pass
        self.root.after(100, self._check_queue) 
    
    def _create_interface(self):
        self.root.geometry("100x100")
        b = tk.Button(self.root, text='Start', command=self.calc)
        b.grid(row=0, column=0)
    
    def calc(self):
        if self.queue.empty():
            self.duh.clear()
            p = Process(target=do_calc, args=[self.queue]) # pass self.queue to do_calc()
            p.start()     
        
def do_calc(q: Queue):
    for i in range(20):
        q.put(i**2)

if __name__ == '__main__':
    q = Queue()
    f = FooUI(q)
    f.start()
acw1668
  • 40,144
  • 5
  • 22
  • 34