-1

I am using multiple threads to work through items in a very large list untell it is empty.

while item_list:
    my_item = item_list.pop()

I check if any items are left in list and if so I pop one and work on it. Is this process thread safe?

Is there chance that when I check there is an item in list but by time I pop it will be gone and raise error? Or any other issues?

John Kugelman
  • 349,597
  • 67
  • 533
  • 578
Matt
  • 57
  • 1
  • 2
  • 9
  • 1
    "Is there chance that when I check there is an item in list but by time I pop" Yes in this case. You can hit the case where two threads have gotten past the "while item_list" but have not yet executed the pop. You should use a [thread lock](https://docs.python.org/3.6/library/threading.html) – IanQ Jan 25 '20 at 01:25
  • I'm curious, what operations are the threads performing? – AMC Jan 25 '20 at 03:45
  • So I surround the code with lock.acquire() and lock.release(). The code is also updating the data it collects into dictionarys. There are no duplicates in item_list and they used as keys in dictionary. So threads may be updating the same dictionary at same time but will never be updating same key at same time. Would that be thread safe? – Matt Jan 25 '20 at 20:21

1 Answers1

1

Yes, a thread-switch could happen between the the two lines and the list could be empty by the time you pop the item. Use a thread-safe queue.Queue() to store your work items.

Mark Tolonen
  • 166,664
  • 26
  • 169
  • 251
  • Out of curiosity: Is `pop` itself thread-safe? So could the threads simply pop without checking first, in a try-except? Or could a thread-switch happen during `pop`'s execution? – Kelly Bundy Jan 25 '20 at 02:38
  • @HeapOverflow Yes, the pop itself is thread-safe, so you could catch the exception if it comes up empty, but see [Are lists thread-safe?](https://stackoverflow.com/a/6319267/235698). Better to use a `queue`. It's difficult to tell what methods are safe. – Mark Tolonen Jan 25 '20 at 06:35