1

I've been learning Python for awhile using official Python site tutorial/library/reference. Today I just accidentally stumbled across formal slicing reference and can't figure out why nobody tells about one possible scenario that should work but unfortunately it doesn't.

Namely, according to this document it should be possible to select elements from the sequence using tuple of indexes and slices:

lst = list(range(1, 100, 2))
slc = slice(10, 20, 3)
print( lst[ 1, 5, 8, slc, 30:40:5, 49 ] )

But as many of you might expect error pops-up

Traceback (most recent call last): File "./slice-test.py", line 68, in print( lst[ 1, 5, 8, slc, 30:40:5, 49 ] ) TypeError: list indices must be integers or slices, not tuple

This notation have been present in Python for a long time. At least the same document for Python 2.x mentions this term as a "extended slicing" here

What am I missing here? I probably misinterpret this notation but I can't figure out where is the catch(except the fact that this is not supported by interpreter of course).

PS I've looked up for an answer elsewhere including this question.

PPS This question is not about indexing or start:end:step slicing per se, so examples of these are not needed.

Chris
  • 26,361
  • 5
  • 21
  • 42
Nikita
  • 85
  • 6
  • 1
    Questions asking why a language does not have a feature are usually considered opinion-based and therefore not suited for Stack Overflow. You might want to reformulated your question. – Klaus D. Apr 20 '21 at 11:28
  • 2
    > why a language does not have a feature This is a little tricky . Official reference mentions this feature but it doesn't accepted by interpreter. So basically language have it but there is a dissonance with interpreter – Nikita Apr 20 '21 at 11:31
  • 4
    @Nikita it *is* accepted by the interpreter. It is *valid syntax*. Similarly, `x[0] = 1` is *valid syntax*. That doesn't mean that `str` objects won't raise an error when you try to use that *valid syntax* on a string. Similarly, slicing with a tuple is valid syntax, but `list` objects reject that. There is no "dissonance". Python has made that syntax available for anyone who wishes to use it in the APIs they design. – juanpa.arrivillaga Apr 20 '21 at 11:38
  • @juanpa.arrivillaga So it's not supported by standard sequences but only by NumPy and other third-party classes/libraries, right? It's not clear why such a decision was made though it answers the question – Nikita Apr 20 '21 at 11:49
  • Probably because it can get complicated very quickly when you talk about overlapping slices etc, and the need probably isn't strong enough to support this kind of complex slicing for regular lists, so the developers opted to keep list slicing simple. – deceze Apr 20 '21 at 11:53
  • @Nikita I don't know specifically about the history of extending slice notation, but more recently, the `@` operator, corresponding to the `__matmul__` special method, was added specifically because of numpy – juanpa.arrivillaga Apr 20 '21 at 11:53
  • @deceze well, I think more fundamentally, list objects are *one dimensional*. Numpy objects have their dimensions and shape baked in. A list can contain *anything*. – juanpa.arrivillaga Apr 20 '21 at 11:54
  • @juanpa Sure, that too, it fundamentally makes a lot more sense with multidimensional numpy objects. You *could* have defined `lst[1, 3, 5:10]` as returning a list containing the second, forth and sixth through eleventh element; but again, that just makes for a complicated implementation, complicated explanation, and probably not enough need to bother with. – deceze Apr 20 '21 at 11:56
  • @deceze ah, yeah, I see what you are saying. It definitely would complicate things. Numpy slicing is actually really complex when you include advanced indexing. Lots of weird rules – juanpa.arrivillaga Apr 20 '21 at 11:58
  • I may have misinterpreted the question, but I don't think this is "why am I not allowed to do it, in this specific case?" (which opens up all the clarifying questions about *what should happen*, and why); but rather "what are the additional restrictions on what can be done, besides the Python grammar?" There is a pretty clear point of confusion: the *cited document seems to suggest* that it can be done, if you are unaware of the distinction between *syntax* and *semantics*. I tried to write an answer based on this. – Karl Knechtel Jun 29 '22 at 00:25
  • See also e.g. https://stackoverflow.com/questions/2936863/implementing-slicing-in-getitem – Karl Knechtel Jun 29 '22 at 00:28

1 Answers1

3

Namely, according to this document it should be possible to select elements from the sequence using tuple of indexes and slices

The document tells you about what is syntactically valid Python code, that can be bytecode-compiled into a .pyc file. It does not tell you about what is semantically valid Python code, that will run without any errors.

Only SyntaxError can be raised by syntactically invalid code. The language grammar only explains how to avoid those exceptions, not any others.

It is allowed to use a tuple of indexes and slices to slice a Python object. It is up to the object's type to define what happens as a result, or if an exception should occur instead.

Python objects handle slicing (and subscription) using the magic method __getitem__. In older versions, this was only for subscription, and a separate __getslice__ would be used for slicing. Either way, if there is only one item between the [] (i.e., no commas in the code), it is passed directly - not as a 1-element tuple. (As explained there: everything that would be a valid subscription is also a valid slicing; these are read by the grammar as subscriptions instead of slicings.) Each slice - written using the special slice syntax, only valid in this slicing context - becomes a slice object, which is passed to __getitem__ just like anything else.

We can make our own class to test it:

>>> class Tester:
...     def __getitem__(self, x):
...         return x
...
>>> t = Tester()
>>> t[0]
0
>>> t['test']
'test'
>>> t[1:2:3]
slice(1, 2, 3)
>>> t[::-1]
slice(None, None, -1)
>>> t[1,2]
(1, 2)
>>> t[(1,2)]
(1, 2)
>>> t[[1,2]]
[1, 2]
>>> t[::-1, ::-1]
(slice(None, None, -1), slice(None, None, -1))
>>> t[1, 5, 8, slice(10, 20, 3), 30:40:5, 49]
(1, 5, 8, slice(10, 20, 3), slice(30, 40, 5), 49)
Karl Knechtel
  • 62,466
  • 11
  • 102
  • 153