0

In my PyQt5 (Python 3.8) application, many QWidget()s get created and destroyed dynamically. Although I pay close attention to the destruction of every QWidget() - something must have slipped. Each cycle, the RAM consumption goes up.

1. The one-liner to disconnect them all

I've read the following blogpost about disconnecting pyqtSignal()s: https://www.sep.com/blog/prevent-signal-slot-memory-leaks-in-python/

At the bottom, the blogpost mentions the following one-liner to disconnect all pyqtSignal()s to/from (?) a QObject():

for x in filter(lambda y: type(y) == pyqtBoundSignal and 0 < element.receivers(y), map(lambda z: getattr(element, z), dir(element))): x.disconnect()

Supposing that element represents a QObject() instance, I wonder if this one-liner will actually destroy all incoming or outgoing signals?

I've tried to rewrite this one-liner in an actual function:

from PyQt5.QtWidgets import *
from PyQt5.QtCore import *
from PyQt5.QtGui import *

def disconnect_signals_connected_to_qobject(qobj:QObject) -> None:
    '''
    Disconnect all signals that are connected to a slot in the given 'qobj'.
    '''
    for x in filter(
            lambda y: type(y) == pyqtBoundSignal and 0 < qobj.receivers(y),
            map(
                lambda z: getattr(qobj, z),
                dir(qobj)
            )
    ):
        x.disconnect()
    return

If the code destroys all incoming signals, then the chosen name disconnect_signals_connected_to_qobject() is correct. Otherwise, I should rename the function into disconnect_signals_emitted_from_qobject(). What's the correct name?

2. Does this really disconnect *all* pyqtSignals?

I still have this nagging feeling that the code will "forget" a few disconnections. Some time ago, I found the following code snippet that claims to disconnect a signal completely:

def discon_sig(signal):
    '''
    Disconnect only breaks one connection at a time,
    so loop to be safe.
    '''
    while True:
        try:
            signal.disconnect()
        except TypeError:
            break
    return

It looks like the one-liner from the blog post forgot the fact that a signal.disconnect() invocation only disconnects one connection at a time. So perhaps, I should rewrite my function like this:

from PyQt5.QtWidgets import *
from PyQt5.QtCore import *
from PyQt5.QtGui import *

def disconnect_signals_connected_to_qobject(qobj:QObject) -> None:
    '''
    Disconnect all signals that are connected to a slot in the given 'qobj'.
    '''
    # Define inner function for a complete signal
    # disconnection
    def discon_sig(signal):
        while True:
            try:
                signal.disconnect()
            except TypeError:
                break
        return

    # Loop to find all signals that need
    # disconnection
    for x in filter(
            lambda y: type(y) == pyqtBoundSignal and 0 < qobj.receivers(y),
            map(
                lambda z: getattr(qobj, z),
                dir(qobj)
            )
    ):
        discon_sig(x)
    return

3. Is disconnecting enough?

I'm getting even more worried after reading this mail thread between a user and the PyQt creator Phil Thompson:

https://www.riverbankcomputing.com/pipermail/pyqt/2019-September/042180.html

Long story short - the user claims that disconnecting pyqtSignal()s is not enough to prevent memory leaks. According to his experiments, it's vital that each pyqtSignal() was bound to a decorated slot. Throw in a few undecorated slot, and eternal doom awaits thee:

Kevin (the 'user' writing to Phil Thompson)

My assumption was that the memory usage from the signal connections wouldn't "build up" (i.e. that memory usage would be independent of the number of SignalObject()s instantiated), which seems to be what happens with the @pyqtSlot() decorated version. In create_slot_objects(), each SlotObject() is instantiated, which connects the signals, and then should be immediately garbage collected, which should disconnect the signals. In the version of the script without the @pyqtSlot() decorators, the number reported for "Memory after slot objects creation" is proportional to the number of SlotObject() instances created, which is why I'd assumed memory had leaked, but I could certainly be thinking through things wrong.

I couldn't find a clear answer from Phil, so I'm now officially paranoid about each and every pyqtSignal() that lives in my application.

4. Is disconnecting *all* signals actually good?

I just performed a test on my application. When browsing through my QGridLayout()'s widgets (in reversed order), I do the following to clean them up:

disconnect_signals_connected_to_qobject(widg)
widg.setParent(None)
widg.deleteLater()
sip.delete(widg)

Despite doing this, I had a memory leak of almost 100 MB per cycle - with one cycle being the filling of my table with 1000 rows and emptying the table again.

Now I omit the pyqtSignal disconnection:

widg.setParent(None)
widg.deleteLater()
sip.delete(widg)

The memory leak dropped to around 20~25 MB per cycle. How is this possible? Maybe the indiscriminate disconnection of all pyqtSignals is hindering a proper cleanup of the QWidget()s involved? Maybe one has to carefully decide what pyqtSignal to keep and which ones to disconnect? Does this mean that the blogpost cited at the start of this question (see https://www.sep.com/blog/prevent-signal-slot-memory-leaks-in-python/) is actually giving bad advice?

K.Mulier
  • 8,069
  • 15
  • 79
  • 141
  • 1
    "a signal.disconnect() invocation only disconnects one connection at a time." AFAIK (as the documentation also says), `signal.disconnect([slot])` "If [slot] is omitted then the signal is disconnected from everything it is connected to.". Can you provide a MRE? – musicamante Jan 29 '21 at 18:55
  • 1
    The first example shown in section 2 is a garbled re-write of code designed for a different purpose. It seems to be based on a misinterpretation of [this answer](https://stackoverflow.com/a/21589403/984421), which does not make the claims you allude to. In fact, it clearly states that the while-loop is needed for safely disconnecting a ***specific handler*** which has been connected to the same signal multiple times. However, in the interests of clarity, I have now amended the answer slightly to avoid any further doubt. – ekhumoro Jan 29 '21 at 22:57
  • I would also like to second the request by musicamante for an MRE. There are many different reasons why a memory leak may occur, so without a concrete test case, an unequivocal answer isn't really possible. – ekhumoro Jan 29 '21 at 23:08
  • Hi @musicamante and @ekhumoro, I would like to give a MRE, but it's a large application and very difficult to extract an MRE from it. However, I just got a notice from a colleague that the memory leak is fixed. Applying a `QStyle()` on a menu - something that we used to do on certain customized `QWidget()`s - seemed to hinder proper garbage collection. We don't know why exactly - maybe that's food for more research or another question. Thanks @ekhumoro for clarifying https://stackoverflow.com/questions/21586643/pyqt-widget-connect-and-disconnect/21589403#21589403 – K.Mulier Jan 30 '21 at 11:50

0 Answers0