2

Assuming the code will be run in a multi threaded environment, is it necessary to implement locks, semaphores, etc. when overriding a magic method? For example, should code like this use a lock to prevent new values from being overwritten by two threads accessing the same missing key?

import time


class TimesDict(dict):
    def __missing__(self, key):
        self[key] = value = [time.time()]
        return value

The confusion comes from the fact that many sources online (ie. http://effbot.org/zone/thread-synchronization.htm#atomic-operations) claim that something like dict_['key'] is atomic, yet the byte code for the given __missing__ example method uses multiple instructions:

dis.dis(TimesDict.__missing__)
  5           0 LOAD_GLOBAL              0 (time)
              2 LOAD_ATTR                0 (time)
              4 CALL_FUNCTION            0
              6 BUILD_LIST               1
              8 DUP_TOP
             10 LOAD_FAST                0 (self)
             12 LOAD_FAST                1 (key)
             14 STORE_SUBSCR
             16 STORE_FAST               2 (value)
  6          18 LOAD_FAST                2 (value)
             20 RETURN_VALUE
Joshua
  • 138
  • 1
  • 7

1 Answers1

1

According to a similar question https://stackoverflow.com/a/17682555/9075002, if the magic method is written in python then it is not thread safe. All python code (and even some C code that releases the GIL) will have concurrency issues. This seems to be in line with this article: https://opensource.com/article/17/4/grok-gil. Comments on the same answer make it clear that it's best to assume nothing is thread safe and the article assures that acquiring a lock is cheap enough to be worth it.

Joshua
  • 138
  • 1
  • 7