PyQt QObject Unexpected Shared State

Question

I spent last night wrangling a particularly confusing bug.

I wrote some proof of concept code to exhibit the issue.

PoC Code

# Standard Imports
import sys

# Import Core Qt modules
from PyQt4.QtGui import QApplication
from PyQt4.QtCore import QObject


class Bar(QObject):
    def __init__(self, parent):
        QObject.__init__(self, parent)
        self.parent = parent


class Foo(QObject):
    items = []

    def __init__(self, parent, item_count=2):
        QObject.__init__(self, parent)
        self.parent = parent

        # Create some items
        for _ in range(item_count):
            self.items.append(Bar(self))

        # Debug
        print "QObject Foo Items: %s" % self.items
        print "QObject Foo Children: %s" % self.children()

# Start up main PyQT Application loop
app = QApplication(sys.argv)

# Build Foo classes
aFoo = Foo(app)
aFoo2 = Foo(app)

# Exit program when our window is closed.
sys.exit(app.exec_())

When this code is executed, here is the output:

QObject Foo Items: [<__main__.Bar object at 0x0234F078>, <__main__.Bar object at 0x0234F0C0>]
QObject Foo Children: [<__main__.Bar object at 0x0234F078>, <__main__.Bar object at 0x0234F0C0>]
QObject Foo Items: [<__main__.Bar object at 0x0234F078>, <__main__.Bar object at 0x0234F0C0>, <__main__.Bar object at 0x0234F150>, <__main__.Bar object at 0x0234F198>]
QObject Foo Children: [<__main__.Bar object at 0x0234F150>, <__main__.Bar object at 0x0234F198>]

Code Explanation

The code is creating two Foo QObject classes. Each class, during __init__, creates some Bar QObject classes and adds them to its class variable self.items.

The bug here is apparently when the second class prints out the items in its self.items class variable there are four and not two Bar classes in there.

The Fix

If you move the class variable definition self.items into the __init__ block of the Foo class, then things work fine.

Thoughts

When items was defined outside of the __init__ call it seems to act like a Singleton and share state across all Foo classes.

Answer 1

While writing this question, I found the answer.

Explanation

When the items variable in the Foo class was defined outside of the __init__ call, the items variable became a static class variable.

Before were done though, there are some gotcha type situations due to how mutable and immutable objects work in Python that you should know about.

Example Assignment

class Bar(object):
    aList = ["Hello World"]
    aString = "Hello World"

    def __init__(self, new=None):
        if new is not None:
            self.aList = new
            self.aString = new[0]

a = Bar()
print a.aList, a.aString
b = Bar(["Goodbye World"])
print a.aList, a.aString
print b.aList, b.aString

Here the class static variables are a list called "aList" and a string called "aString". We created two Bar classes. The first class doesn't touch the static variables, but the second does. The second Bar class sets the static variables to new objects passed into the __init__ call.

# Output
['Hello World'] Hello World
['Hello World'] Hello World
['Goodbye World'] Goodbye World]

This is very understandable output. The static variables were not changed on the class, as seen by the ["Hello World"] Hello World printing twice. Instead, the second Bar class simply set its own "aList" and "aString" variables as a new list and string object in its class self object and this overrode the class' static variables for this specific Bar class causing us to see ["Goodbye World"] instead.

Example Modification

class Bar(object):
    aList = []
    aString = "A"

    def __init__(self, append=False):
        if append:
            self.aList.append('A')
            self.aString += 'A'

a = Bar()
print a.aList, a.aString
b = Bar(append=True)
print a.aList, a.aString
print b.aList, b.aString

Same as before but instead of setting the "aList" and "aString" variables to a new object we are simply concatenating onto them.

# Output
[] A
['A'] A
['A'] AA

Uh oh! What happened?

Our string class static variable "aString" came out as expected. We started it with just "A" and it stayed constant after the second instantiation of Bar. When the second Bar class print out its "aString" variable we saw "AA". This is due to the command self.aString += 'A'. This code takes the current value of aString, which is the Bar class static variable at this point, and appends another "A" before reassigning it to self.aString but now it is in the scope of __init__ and is thus set in the class' self object and as we saw above this now overrides the static variable aString for this specific Bar class.

Our list class static variable came out unexpected. Instead of the same behavior as we saw with our string variable, our static list variable changed during the instantiation of the second Bar class. The change saw "A" added to our list object. The reason this happened is because unlike our static string object, when we execute self.aList.append('A') there is not re-assignment of the list variable into the executing class' self object. append() is accessing the list directly with no re-assignment. With no re-assignment there is no scope change to the static variable and thus no overriding will happen.

Conclusion

The string object we saw in our examples is immutable in Python. When we appended to it, we were not adding to the string object in place. Instead, we were making a new string with an added character and then re-assigning our static variable to this new string object. This re-assignment coupled with being in the scope of __init__ set the new string variable to the class' self object and overrode the static string variable altogether for the class.

The list object is mutable in Python. When we appended to it we did modify the list in place. Due to this, there was no re-assignment and the scope of the static variable did not change from being a static class variable. Due to this all changes to the variable are reflected in all classes as we saw in our examples.

Links for More Reading

Read this question and this question to learn more about static class variables in Python.

More information on mutable/immutable objects in Python can be read here and here (external link).