0

I'm using a couple of class attributes to keep track of aggregate task completion across multiple instances of class. When reading or updating the class attributes do I need to use a lock of some sort?

class ClassAttrExample:
    
    of_type_list = []
    of_type_int = 0
    
    def __init__(self, name):
        self.name = name
        
    def do_task(self):
        # does some stuff
        # do I need a lock context here???
        self.of_type_list.append(self.name)
        self.of_type_int += 1
Conner M.
  • 1,954
  • 3
  • 19
  • 29
  • 1
    Is it multithreaded? Then no. This code doesn't work by the way. Needs `self.of_type_xx`. – Mark Tolonen Jul 12 '22 at 23:32
  • 1
    @MarkTolonen: More subtly, the `of_type_int` case needs `type(self).of_type_int += 1`. If it uses `self.of_type_int += 1`, then on the first execution (for each instance) it will read the class attribute, increment it, then store it to a new instance attribute that shadows the class attribute for that instance from then on. The class attribute would never be incremented, and each instance would have its own separate count. – ShadowRanger Jul 12 '22 at 23:46
  • @ShadowRanger Yes I know. Python does have its warts. Better to reference the class when accessing class variables. – Mark Tolonen Jul 13 '22 at 00:50

1 Answers1

1

If not threads are involved, no locks are required just because class instances share data. As long as the operations are performed in the same thread, everything is safe.

If threads are involved, you'll want locks.

For the specific case of CPython (the reference interpreter), as an implementation detail, the .append call does not require a lock. The GIL can only be switched out between bytecodes (or when a bytecode calls into C code that explicitly releases it, which list never does), and list.append is effectively atomic as a result (all the work it does occurs within a single CALL_METHOD bytecode which never calls back into Python level code, so the GIL is definitely held the whole time).

By contrast, += involves reading the input operand, then performing the increment, then reassigning the input, and the GIL can be swapped between those operations, leading to missed increments when two threads read the value before either writes back to it.

So if multithreaded access is possible, for the int case, the lock is required. And given you need the lock anyway, you may as well lock around the append call too, ensuring the code remains portable to GIL-free Python interpreters.

A fully portable thread-safe version of your class would look something like:

import threading

class ClassAttrExample:
    _lock = threading.Lock()    
    of_type_list = []
    of_type_int = 0
    
    def __init__(self, name):
        self.name = name
        
    def do_task(self):
        # does some stuff
        with self._lock:
            # Can't use bare name to refer to class attribute, must access
            # through class or instance thereof
            self.of_type_list.append(self.name)  # Load-only access to of_type_list
                                                 # can use self directly
            type(self).of_type_int += 1  # Must use type(self) to avoid creating
                                         # instance attribute that shadows class
                                         # attribute on store
ShadowRanger
  • 143,180
  • 12
  • 188
  • 271
  • Even if the GIL protected the `+=` operation, it wouldn't protect the *combination* of the `append` and the `+=` - another thread could intervene between those operations and see the list and the int in a state inconsistent with each other. That's another commonly overlooked reason locks are important. A sequence of two atomic operations is not atomic. – user2357112 Jul 12 '22 at 23:44
  • Also, if you call `do_task` on an instance of a subclass of this class, `type(self).of_type_int += 1` is going to try to update `of_type_int` on the subclass instead of on `ClassAttrExample`. – user2357112 Jul 12 '22 at 23:48
  • @user2357112: Yep. The OP's example isn't clear on whether the two operations should be considered one atomic operation, but if there are, then GIL or no-GIL doesn't matter, you need the lock. – ShadowRanger Jul 12 '22 at 23:49
  • @user2357112: True. If you want subclasses to share the same attribute instead of maintaining their own (not necessarily what you'd want), you could (assuming you're not still on Python 2) use `__class__.of_type_int += 1` to avoid repeating the class name (that's a little magical, but the availability of bare `__class__` referring to the class the function was defined in is a language guarantee provided to support the no-arg `super()` calls that just happens to be useful here as well). – ShadowRanger Jul 12 '22 at 23:52