Python dictionary needs synchronization?

Question

If a have a Python dictionary and have multiple threads adding to it, do I have to synchronize the addition of elements if I know that every thread is going to add to its own key. ( What I mean is that no addition for the same key can take place at the same time ).

possible dup of http://stackoverflow.com/questions/3358770/python-dictionary-is-thread-safe — shx2, Mar 11 '13 at 10:16
You're right. But it is a duplicate of http://stackoverflow.com/questions/6953351/thread-safety-in-pythons-dictionary — shx2, Mar 11 '13 at 10:21
@shx2: It's just similar. In http://stackoverflow.com/questions/6953351/thread-safety-in-pythons-dictionary, the dictionary structure is static (no new keys added in threads), so the access is usually safe (in the end, just a single pointer reference is updated). Here, the structure of the dict changes — Aaron Digulla, Mar 11 '13 at 10:27

Martijn Pieters · Accepted Answer · 2013-03-11T10:39:39.997

You may need to synchronize access to a dictionary object when your keys are not unique per thread. Access is not necessarily thread safe.

You can always use the dis (disassembler) module to verify any python code; python thread context can switch between any opcode (opcodes themselves are executed atomically); access to a dict can be spread over several opcodes:

>>> def foo():
...     somedict = {}
...     somedict['bar'] = 'spam'
... 
>>> import dis
>>> dis.dis(foo)
  2           0 BUILD_MAP                0
              3 STORE_FAST               0 (somedict)

  3           6 LOAD_CONST               1 ('spam')
              9 LOAD_FAST                0 (somedict)
             12 LOAD_CONST               2 ('bar')
             15 STORE_SUBSCR        
             16 LOAD_CONST               0 (None)
             19 RETURN_VALUE

There are 4 opcodes involved with the somedict['bar'] = spam line, loading the somedict reference, loading the 'bar' and 'spam' constants, and calling the store operation. Thread contexts can be switched between any of these opcodes.

That said, if your keys are guaranteed to be unique per thread, you could get away with not locking the structure; like all opcodes, the STORE_SUBSCR opcode is executed by the python interpreter under the GIL, so multiple threads storing in and reading information from their own key should be safe.

The Python standard library library does this in a few locations; using thread.thread_ident() as a dictionary key to store per-thread information in a dictionary. The threading module for example, keeps a _active module-global dictionary keyed on thread identifiers.

The `STORE_SUBSCR` opcode isn't necessarily atomic: if overwriting a key that already exists, and either the key or the value that is overwritten uses `__del__` (or has the last reference to something that does) you can get a thread switch when objects are released. In practice this isn't likely to be a problem, but it is worth remembering when you try to take advantage of the GIL to avoid explicit locking. — Duncan, Mar 11 '13 at 13:32

Python dictionary needs synchronization?

1 Answers1