4

We have a list to which data is appended at regular time intervals and this procedure takes time so using usual mutex to protect the entire list during writes is not the most efficient solution. How to organize reads and writes to such list in a more concurrent fashion?

Pavlo
  • 1,594
  • 2
  • 15
  • 30

1 Answers1

10

You don't need locking when using list.append() or list.extend() with multiple threads. These operations are thread-safe.

Here is a brief overview of operations that are thread-safe: https://docs.python.org/3/faq/library.html#what-kinds-of-global-value-mutation-are-thread-safe

It's also worth mentioning that from a performance standpoint it's much faster to prepare sub-lists in separate threads, and then extend the main list with these sub-lists.

leovp
  • 4,528
  • 1
  • 20
  • 24
  • 1
    correct me if im wrong, python lists are implemented with arrays, not dynamic structures. So when list is full a new allocation is required. But how can threads be informed about that case? Threads may try to append a data to a full list. I am wondering how that safety is provided. – Ali Berat Çetin Aug 12 '20 at 13:28
  • 1
    I think it's because of the global interpreter lock. The Python VM will not execute code from different threads at the same time, instead it quickly switches back & forth between them. – Myles Hollowed Jan 22 '21 at 19:09
  • They're thread-safe. Does that mean they're also multiprocessing-safe? – falsePockets Oct 25 '21 at 00:30