3

After having read various articles that explain GIS and threads in Python, and Are locks unnecessary in multi-threaded Python code because of the GIL? which is a very useful answer, I have one "last question".

If, ideally, my threads only operate on shared data through atomic (Python VM) instructions, e.g. appending an item to a list, a lock is not needed, right?

Community
  • 1
  • 1
giohappy
  • 520
  • 1
  • 5
  • 17
  • 5
    Don't use link shorteners. Also, links to other questions on this site automatically are replaced by the title of the question. – agf Aug 17 '11 at 20:31

4 Answers4

2

That really depends on your application. You may need locks for your specific use case just like in any other language, but you don't need to protect the python objects getting corrupted anyway. In that sense you don't need locks.

Here's an example that uses a bunch of pretty much atomic operations, but can still behave in unexpected ways when you combine them.

Thread 1:

v = l[-1]
DoWork(v]
del l[-1]

Thread 2:

l.append(3)

If Thread 2 runs in between the first and last statement of Thread 1, then Thread 1 just deleted the wrong work item. None of the python objects are corrupt or anything, but you still got an unexpected result, and potentially exceptions could be thrown.

If you have shared data structures, you usually need to protect them with locks, or even better use already written protected versions, like in this case maybe a Queue: http://docs.python.org/library/queue.html

user12861
  • 2,358
  • 4
  • 23
  • 41
0

In theory not, but it depends on the logic, you need a lock when you are preserving order, for example.

orlp
  • 112,504
  • 36
  • 218
  • 315
0

When you share data between threads, you should always make sure your data is properly synchronized because you cannot rely on wether operations will be or not atomic in the future.

It is easier to get the multi-threading design right in the first place than try to fix something that breaks because of a change in implementation or a wrong assumption.

Martin
  • 5,954
  • 5
  • 30
  • 46
0

Thanks everyone for the answers! It's clear that thread sync requirements are bound to the application logic, but I can rely on the GIL to do not corrupt the builtin objects internals (is the operations are atomic). It wasn't clear to me when the GIL i said to protect the "internal state" of the interpreter, its internal data structures... I mean, this is an effect, but the GIL protects every allocated builtin structure, both for objects created and used by internal operations of the interpreter and objects created by the application. That was my doubt.

PS: I'm sorry for having answered so late, but I didn't receive email notifies...

giohappy
  • 520
  • 1
  • 5
  • 17