Is Python list slicing thread safe?

Question

Coming from Are lists thread-safe?, I need to know if specifically list slicing is thread safe. It's not clear to me from the linked article What kinds of global value mutation are thread-safe?.

Based on the answer to is list.pop thread safe in python, it seems I really need to know if list slicing an atomic operation.

I have a list some_list that another thread is continuously appending to. In the main thread, I sometimes peek (without poping) at:

The last element of the list like so: some_list[-1] (indexing)
Last couple of elements of the list like so: some_list[-3:] (slicing)

So, the question is, in CPython (I use Python 3.10), is list slicing thread-safe?

List slicing is essentially a loop over the range of indexes. If `pop()` isn't atomic, I can't imagine why that would be. — Barmar, Mar 30 '22 at 20:12
Note that list slicing doesn't modify the list being sliced, it just reads from it, so none of your links to questions about modification are relevant. But it's reading multiple indexes, and if some other thread writes to those indexes, you'll get an inconsistent slice. — Barmar, Mar 30 '22 at 20:13
Do you care about slicing (creating a new list with a subset of the elements of another list) or indexing (accessing a single element of a list by index)? While more or less the same, there's a chance that the threading implications are different — Brian61354270, Mar 30 '22 at 20:16
@Brian I had said slicing to be more conservative, though most of my use cases are just indexing to get the last element via `some_list[-1]` — Intrastellar Explorer, Mar 30 '22 at 23:24
@Barmar the linked article on `pop` mentions that it's atomic, so it's likely thread safe. In my particular use case, the other thread is just `append`ing to the list, not modifying values. So do you think reading elements from the list can be considered thread safe? — Intrastellar Explorer, Mar 30 '22 at 23:27
Oh, I misread it. I thought his answer "YES" was to "Will it ever be the case that data is lost due to race condition between threads" — Barmar, Mar 30 '22 at 23:29
_via slicing like so: `some_list[-1]`_ <-- this is not slicing. Please clarify whether you need slicing or indexing. — wim, Apr 08 '22 at 18:09
Yes @wim I have clarified, and my apologies for a misunderstanding when I authored. Really, I use both, so an answer addressing both slicing/indexing is most desirable — Intrastellar Explorer, Apr 08 '22 at 21:34

kosciej16 · Accepted Answer · 2022-04-09T08:58:58.120

tl;dr both indexing and slicing are thread safe, but slicing consist of building slice and applying slice. Between them thread can modify our list.

In stackoverflow page you posted you can find a list of atomic operations for iterables and both, indexing and slicing are there.

Lets look at bytecode for slicing:

import dis

l = [1, 2, 3]


def f():
    l[1:3]


dis.dis(f)

  7           0 LOAD_GLOBAL              0 (l)
              2 LOAD_CONST               1 (1)
              4 LOAD_CONST               2 (3)
              6 BUILD_SLICE              2
              8 BINARY_SUBSCR
              (...)

BINARY_SUBSCR is a moment when you access the slice and it's indeed single operation. Check that:

import time
import threading

l = [0] * 1000000000


def append():
    start = time.perf_counter()
    time.sleep(0.1)
    l.append(1)  # Doesn't execute until main thread access slice
    print("append", time.perf_counter() - start)  # 3.4184917740058154


t = threading.Thread(target=append)
t.start()
start = time.perf_counter()
x = l[:]  # Reading via slicing
print("main", time.perf_counter() - start)  # 3.418436762993224
print("last_elem", x[-1])  # 0
t.join()

So fear not of changes in list while accessing a slice (and same with accessing an index) as it's simple opcode and is protected by GIL.

What can happen then?

# l = [1, 2, 3]
x = l[0:3]

You cannot assume x will contain all list.

We first (BUILD_SLICE) with start 0, end 3.
Then another thread can modify list e.g. by appending element.
Only then we access slice with thread safe way.

Can it make our program crash? I doubt, as using out of bounds slice is safe.

Thank you @kosciej16. To me from your code snipped it seems like the slice operation is thread safe, as the `append` takes place afterwards. Would you mind elaborating more? Also, can you explain why indexing can be considered thread safe? — Intrastellar Explorer, Apr 08 '22 at 23:10
@IntrastellarExplorer Updated an answer, let me know if it's more clear now. — kosciej16, Apr 09 '22 at 08:59
Yes @kosciej16 it is much clearer, thank you for editing several times! I have two questions. In the final sentence, if another thread as appended to `l`, how would `x` risk being out of bounds? It still seems like all the indices are still there. The other question I have is, what is taking place inside `BUILD_SLICE`? From reading the `dis` docs on `BUILD_SLICE` (I don't know assembly code), it seems to be temporarily saving the slice's indices, is that correct? — Intrastellar Explorer, Apr 09 '22 at 23:42
1) It can't in case of appending, but another thread can remove element from list l as well. 2) You can assume it works the same as `slice` constructor - we pop arguments from stack and push slice object. The difference is the same as calling {0} (which opcode is BUILD_SET) vs set() (which searches for global named `set` and call it, what is slower) — kosciej16, Apr 10 '22 at 18:16

Is Python list slicing thread safe?

1 Answers1