512

As we all know, there's list comprehension, like

[i for i in [1, 2, 3, 4]]

and there is dictionary comprehension, like

{i:j for i, j in {1: 'a', 2: 'b'}.items()}

but

(i for i in (1, 2, 3))

will end up in a generator, not a tuple comprehension. Why is that?

My guess is that a tuple is immutable, but this does not seem to be the answer.

Shady Xu
  • 5,527
  • 2
  • 14
  • 18

13 Answers13

746

You can use a generator expression:

tuple(i for i in (1, 2, 3))

but parentheses were already taken for … generator expressions.

Tom Zych
  • 13,329
  • 9
  • 36
  • 53
Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • 36
    By this argument, we could say a list-comprehension is unnecessary too: `list(i for i in (1,2,3))`. I really think it's simply because there isn't a clean syntax for it (or at least nobody has thought of one) – mgilson Jun 05 '13 at 13:00
  • 113
    A list or set or dict comprehension is just syntactic sugar to use a generator expression that outputs a specific type. `list(i for i in (1, 2, 3))` is a generator expression that outputs a list, `set(i for i in (1, 2, 3))` outputs a set. Does that mean the comprehension syntax is not needed? Perhaps not, but it is awfully handy. For the rare cases you need a tuple instead, the generator expression will do, is clear, and doesn't require the invention of another brace or bracket. – Martijn Pieters Jun 05 '13 at 13:02
  • 31
    The answer is obviously because tuple syntax and parenthesis are ambiguous – Charles Salvia Jun 05 '13 at 13:14
  • 4
    @MartijnPieters Wanted to point out that `list(generator expression)` isn't exactly the same as `[generator expression]`: http://www.thecodingforums.com/threads/generator-expressions-vs-comprehensions.723967/ – JKillian Sep 28 '14 at 05:35
  • 1
    @JKillian: the difference exists but is too subtle for the vast majority to have to care about. Playing with an iterator like that in the expression without handling the `StopIterator` exception is going to be rare enough. :-) – Martijn Pieters Sep 28 '14 at 09:02
  • 2
    @MartijnPieters Good point, although I stumbled across it the other day while trying to write my own `zip` function as an exercise to learn Python. Needless to say, it confused the heck out of me why `tuple(gen exp)` failed but `tuple([gen exp])` worked perfectly. Anyways, good to have it noted here that `StopIteration` will be swallowed by generator expressions/comprehensions but will propagate out of list/set/dictionary comprehensions. – JKillian Sep 28 '14 at 18:31
  • 43
    The difference between using a comprehension and using a constructor+generator is more than subtle if you care about performance. Comprehensions result in faster construction compared to using a generator passed to a constructor. In the latter case you are creating and executing functions and functions are expensive in Python. `[thing for thing in things]` constructs a list much faster than `list(thing for thing in things)`. A tuple comprehension would not be useless; `tuple(thing for thing in things)` has latency issues and `tuple([thing for thing in things])` could have memory issues. – Justin Turner Arthur Sep 12 '16 at 22:40
  • 19
    @MartijnPieters, Can you potentially reword `A list or set or dict comprehension is just syntactic sugar to use a generator expression`? It's causing [confusion](https://stackoverflow.com/questions/52026406/are-python3-5-tuple-comprehension-really-this-limited) by people seeing these as *equivalent* means to an end. It's not technically syntactic sugar as the processes are actually different, even if the end product is the same. – jpp Aug 26 '18 at 19:31
  • 2
    @jpp: that's in a *comment*, not in my answer. Comments are generally not editable. Technically I can edit mine still, but only because I am a moderator. And I stand by my comment, as the *syntax is very close. Decorators are also syntactic sugar, and their implementation differs in important ways from the syntax they replaced, so this is not an isolated example. I am not convinced that one example of confusion equals general confusion. – Martijn Pieters Aug 27 '18 at 13:47
  • 3
    Related: [Are list comprehensions syntactic sugar for `list(generator expression)` in Python 3?](https://stackoverflow.com/questions/30096351/are-list-comprehensions-syntactic-sugar-for-listgenerator-expression-in-pyth) – jpp Aug 28 '18 at 09:02
105

Raymond Hettinger (one of the Python core developers) had this to say about tuples in a recent tweet:

#python tip: Generally, lists are for looping; tuples for structs. Lists are homogeneous; tuples heterogeneous. Lists for variable length.

This (to me) supports the idea that if the items in a sequence are related enough to be generated by a, well, generator, then it should be a list. Although a tuple is iterable and seems like simply a immutable list, it's really the Python equivalent of a C struct:

struct {
    int a;
    char b;
    float c;
} foo;

struct foo x = { 3, 'g', 5.9 };

becomes in Python

x = (3, 'g', 5.9)
Nathaniel Ford
  • 20,545
  • 20
  • 91
  • 102
chepner
  • 497,756
  • 71
  • 530
  • 681
  • 46
    The immutibility property can be important though and often a good reason to use a tuple when you would normally use a list. For example, if you have a list of 5 numbers that you want to use as a key to a dict, then tuple is the way to go. – pavon Dec 07 '15 at 18:56
  • 2
    Thats a nice tip from Raymond Hettinger. I would still say there is a use case for using the tuple constructor with a generator, such as unpacking another structure, perhaps larger, into a smaller one by iterating over the attrs that you are interested in converting to a tuple record. – dave Jul 01 '16 at 13:37
  • 2
    @dave You can probably just use `operator.itemgetter` in that case. – chepner Jul 01 '16 at 13:39
  • @chepner, I see. That is pretty close to what I mean. It does return a callable so if I only need to do it once I don't see much of a win vs just using `tuple(obj[item] for item in items)` directly. In my case I was embedding this into a list comprehension to make a list of tuple records. If I need to do this repeatedly throughout the code then itemgetter looks great. Perhaps itemgetter would be more idiomatic either way? – dave Jul 01 '16 at 16:50
  • 3
    I see the relationship between frozenset and set analogous to that of tuple and list. It's less about heterogeneity and more about immutability - frozensets and tuples can be keys to dictionaries, lists and sets cannot due to their mutability. – polyglot Feb 01 '18 at 11:52
  • 2
    There's also a common case where you use a generator to produce a struct-like thing: where you're processing text records such as CSV. This is often written `line_values = tuple(int(x.trim()) for x in line.split(','))`. As others have noted, using the tuple constructor here instead of a comprehension has performance implications, and parsing large datasets of this type is a case where you really care about performance. – Tom Jul 06 '18 at 11:45
102

Since Python 3.5, you can also use splat * unpacking syntax to unpack a generator expression:

*(x for x in range(10)),
wovano
  • 4,543
  • 5
  • 22
  • 49
czheo
  • 1,771
  • 2
  • 12
  • 22
  • 4
    This is great (and it works), but I can't find anywhere it's documented! Do you have a link? – felixphew Dec 24 '17 at 05:47
  • 26
    Note: As an implementation detail, this is basically the same as doing `tuple(list(x for x in range(10)))` ([the code paths are identical](https://github.com/python/cpython/blob/3.7/Python/ceval.c#L2283), with both of them building a `list`, with the only difference being that the final step is to create a `tuple` from the `list` and throw away the `list` when a `tuple` output is required). Means that you don't actually avoid a pair of temporaries. – ShadowRanger Oct 02 '18 at 19:10
  • 12
    To expand on the comment of @ShadowRanger, [here's a question](https://stackoverflow.com/questions/47573416/tuple-comprehensions-and-the-star-splat-unpack-operator) where they show that the splat+tuple literal syntax is actually quite a bit slower than passing a generator expression to the tuple constructor. – Lucubrator Nov 21 '18 at 08:39
  • 2
    I'm trying this in Python 3.7.3 and `*(x for x in range(10))` doesn't work. I get `SyntaxError: can't use starred expression here`. However `tuple(x for x in range(10))` works. – Ryan H. Sep 14 '19 at 00:17
  • 20
    @RyanH. you need put a comma in the end. – czheo Sep 15 '19 at 01:02
  • Just to add to comments, this is **just a tuple expansion**, since the tuple contains a generator expression, **the gets expanded/unpacked while displaying**. since tuple can be written without parentheses but just a comma, it works – mahee96 Sep 26 '21 at 06:12
  • 1
    ex: `*range(10),` **NOTE: The comma at the end to indicate it is a tuple** – mahee96 Sep 26 '21 at 06:12
  • *NOTE:* `tuple(x for x in range(10))` works coz the inner expression is a generator expression and it first returns an iterator, which is a single argument for the tuple, so the tuple is happy and uses the iterator of the to iterate over the items and create the final tuple. But for `tuple(*range(10))` the range is unpacked and tuple constructor receives 10 items which it doesn't like and says: `tuple expected at most 1 arguments, got 10`. so the correct syntax for constructor is `tuple()`. Hence using `tuple(range(10))` works. – mahee96 Sep 26 '21 at 06:19
73

As another poster macm mentioned in his answer, the fastest way to create a tuple from a generator is tuple([generator]).


Performance Comparison

  • List comprehension:

      $ python3 -m timeit "a = [i for i in range(1000)]"
      10000 loops, best of 3: 27.4 usec per loop
    
  • Tuple from list comprehension:

      $ python3 -m timeit "a = tuple([i for i in range(1000)])"
      10000 loops, best of 3: 30.2 usec per loop
    
  • Tuple from generator:

      $ python3 -m timeit "a = tuple(i for i in range(1000))"
      10000 loops, best of 3: 50.4 usec per loop
    
  • Tuple from unpacking:

      $ python3 -m timeit "a = *(i for i in range(1000)),"
      10000 loops, best of 3: 52.7 usec per loop
    

My version of python:

$ python3 --version
Python 3.6.3

So you should always create a tuple from a list comprehension unless performance is not an issue.

Tom
  • 839
  • 6
  • 6
  • 35
    Note: `tuple` of listcomp requires a peak memory usage based on the combined size of the final `tuple` and `list`. `tuple` of a genexpr, while slower, does mean you only pay for the final `tuple`, no temporary `list` (the genexpr itself occupying roughly fixed memory). Usually not meaningful, but it can be important when the sizes involved are huge. – ShadowRanger Oct 02 '18 at 19:03
  • 1
    Very informative. Tuple from a generator would not be the best choice in this case. I think `tuple([i for i in range(1000)])` is the best in terms of readability and speed. Though ofc, not sure of the timings on smaller / bigger / different datasets – jamylak Mar 04 '21 at 03:01
  • 1
    when I tried tuple from list comprehension v/s tuple from generator with bigger data (roughly say range(1_000_000)) you'll see tuple from generator will take less time although it's not so significant but you'll end up saving both size and time in case of bigger data – Utsav Patel Apr 15 '21 at 19:40
37

Comprehension works by looping or iterating over items and assigning them into a container, a Tuple is unable to receive assignments.

Once a Tuple is created, it can not be appended to, extended, or assigned to. The only way to modify a Tuple is if one of its objects can itself be assigned to (is a non-tuple container). Because the Tuple is only holding a reference to that kind of object.

Also - a tuple has its own constructor tuple() which you can give any iterator. Which means that to create a tuple, you could do:

tuple(i for i in (1,2,3))
Inbar Rose
  • 41,843
  • 24
  • 85
  • 131
  • 10
    In some ways I agree (about it not being necessary because a list will do), but in other ways I disagree (about the reasoning being because it's immutable). In some ways, it makes more sense to have a comprehension for immutable objects. who does `lst = [x for x in ...]; x.append()`? – mgilson Jun 05 '13 at 12:49
  • @mgilson I am not sure how that relates to what I said? – Inbar Rose Jun 05 '13 at 12:53
  • 3
    @mgilson if a tuple is immutable that means the underlying implementation *cannot* "generate" a tuple ("generation" implying building one piece at a time). immutable means you can't build the one with 4 pieces by altering the one with 3 pieces. instead, you implement tuple "generation" by building a list, something designed for generation, then build the tuple as a last step, and discard the list. The language reflects this reality. Think of tuples as C structs. – Scott Feb 21 '15 at 17:56
  • 2
    although it would be reasonable for the syntactic sugar of comprehensions to work for tuples, since you cannot use the tuple until the comprehension is returned. Effectively it does not act like mutable, rather a tuple comprehension could behave much like string appending. – uchuugaka Oct 22 '17 at 12:33
  • What you do when `(1,2,3)` isn't easy enough. – Ryan Apr 15 '22 at 16:32
15

My best guess is that they ran out of brackets and didn't think it would be useful enough to warrent adding an "ugly" syntax ...

mgilson
  • 300,191
  • 65
  • 633
  • 696
  • 1
    Angle brackets unused. – uchuugaka Oct 22 '17 at 12:34
  • @uchuugaka -- Not completely. They're used for comparison operators. It could probably still be done without ambiguity, but maybe not worth the effort ... – mgilson Oct 22 '17 at 21:57
  • No, that is very different. The language grammar is very specific about those as tokens. They have a very clear semantic and lexical scope that would be unambiguous (or nearly enough to make it work with minor changes) if also applied to new tokens like exist for the other brackets. Parens are already used to delineate scope of lots of different things, so it should be very very doable. Were it only that somebody decided to. Honestly, sets also need some literal love and comprehensions should get sigils. – uchuugaka Oct 23 '17 at 08:01
  • 7
    @uchuugaka Worth noting that `{*()}`, though ugly, works as an empty set literal! –  Apr 11 '18 at 17:54
  • Absolutely HIDEOUS. I like it, but only because it is sick obscure stuff. Nobody should ever use that. Awesome. – uchuugaka Apr 13 '18 at 02:55
  • What version of python is that supposed to work in ? – uchuugaka Apr 13 '18 at 02:56
  • @M.I.Wright, I'd call that the Cyclops (sideways). Does it have a name? – Quantum Mechanic Sep 30 '18 at 09:17
  • @QuantumMechanic Nope, no common name -- likely because it's not often used (and shouldn't be at all used!). From a purely-aesthetic standpoint, though, I admit I'm somewhat partial now to `{*''}` –  Sep 30 '18 at 13:26
  • 4
    Ugh. From an aesthetic standpoint, I think I'm partial to `set()` :) – mgilson Oct 01 '18 at 17:30
  • @QuantumMechanic: I came up with `{*()}` almost immediately after PEP 448 came out, and I've been calling it the one-eyed monkey operator. I doubt it's the only name people have come up with. – ShadowRanger Oct 02 '18 at 19:05
  • @ShadowRanger: It turns out that these all evaluate to the empty set: `{*''}, {*""}, {*()}, {*[]}, {*{}}`. So inadvertently, TIMTOWTDI. I guess I like `{*[]}` for it's appearance as a posh **Letterbox in Kent**. – Quantum Mechanic Oct 03 '18 at 10:24
  • 4
    @QuantumMechanic: Yeah, that's the point; the unpacking generalizations made the empty "set literal" possible. Note that `{*[]}` is strictly inferior to the other options; the empty string and empty `tuple`, being immutable, are singletons, so no temporary is needed to construct the empty `set`. By contrast, the empty `list` is not a singleton, so you actually have to construct it, use it to build the `set`, then destroy it, losing whatever trivial performance advantage the one-eyed monkey operator provides. – ShadowRanger Oct 03 '18 at 10:54
13

Tuples cannot efficiently be appended like a list.

So a tuple comprehension would need to use a list internally and then convert to a tuple.

That would be the same as what you do now : tuple( [ comprehension ] )

macm
  • 131
  • 1
  • 3
3

Parentheses do not create a tuple. aka one = (two) is not a tuple. The only way around is either one = (two,) or one = tuple(two). So a solution is:

tuple(i for i in myothertupleorlistordict) 
Massimiliano Kraus
  • 3,638
  • 5
  • 27
  • 47
ilias iliadis
  • 601
  • 8
  • 15
  • `one = (two,)` and `one = tuple(two)` do not evaluate to the same value. The argument to `tuple` must be an iterator. `one = (two,)` is equivalent with `one = tuple(i for i in two)`, `one = tuple((two,))`, and `one = tuple([two])`. – markusk Jan 03 '22 at 11:55
1

I believe it's simply for the sake of clarity, we do not want to clutter the language with too many different symbols. Also a tuple comprehension is never necessary, a list can just be used instead with negligible speed differences, unlike a dict comprehension as opposed to a list comprehension.

jamylak
  • 128,818
  • 30
  • 231
  • 230
  • 1
    "*Also a tuple comprehension is never necessary, a list can just be used instead with negligible speed differences*" Calling C++ libraries with a list instead of a tuple may return an error. However it's not that difficult to convert the list into a tuple by `tuple(list)` – mins Mar 03 '21 at 13:20
  • 1
    @mins That appears to be the best option you can choose from here https://stackoverflow.com/a/48592299/1219006 based on timing – jamylak Mar 04 '21 at 03:00
0

On my python (3.5) using a generator with deque from collections is slightly quicker then using a list comprehension:

>>> from collections import deque
>>> timeit.timeit(lambda: tuple([i for i in range(10000000)]),number=10)
9.294099200000005
>>> timeit.timeit(lambda: tuple(deque((i for i in range(10000000)))),number=10)
9.007653800000014
B.R.
  • 234
  • 2
  • 7
  • I did not see the speed advantage. Did you try to repeat the timing multiple times? The results can vary considerably on repeated runs. – pabouk - Ukraine stay strong Jun 14 '22 at 08:21
  • I just checked on python 3.5 and I could reproduce it. But this might be different for other python versions. It seams to be somehow plausible because deque does not need the index related overhead a list has. – B.R. Jun 14 '22 at 12:22
  • Just for information: in Python 3.10.4 I am getting values around 6 for the list variant, 8 for the deque variant. The variation between individual runs is bigger than the 0.3 seconds difference between your results. I am running it inside WSL2 and the virtualization can possibly cause the large variation. – pabouk - Ukraine stay strong Jun 14 '22 at 12:50
  • 1
    I rechecked now with python 3.9.2 on same computer like before and I got: First case: 5.848282200000003 and second case: 6.6902867000000015 I guess the implementation related to `list` is more improved compared with `deque` This means my statement is only valid for older python versions – B.R. Jun 14 '22 at 16:48
0

Because you can not append items to a tuple. This is how a simple list comprehension can be converted into more basic python code.

_list = [1,2,3,4,5]
clist = [ i*i for i in _list ]
print(clist)

clist1 = []
for i in _list:
    clist1.append(i*i)
print(clist1)

Now using a tuple comprehension for above example means appending items into a tuple which is not allowed. Though you can covert this list to a tuple once it is ready by using tuple(clist1)

KnowledgeSeeeker
  • 620
  • 1
  • 9
  • 14
-1

Well there is tuple comprehension in python3 now. You can follow below code snippet.

(k*k for k in range(1,n+1)) 

it will return a generator object comprehension.

Ankit Tiwari
  • 4,438
  • 4
  • 14
  • 41
Arun
  • 3,440
  • 1
  • 11
  • 19
  • 3
    That is not correct. This does not result in a tuple, but in a generator. If you try this `type( (k*k for k in range(1,11)) )`, you will see that it returns `` – Glenn Mackintosh Jan 22 '23 at 21:00
-5

We can generate tuples from a list comprehension. The following one adds two numbers sequentially into a tuple and gives a list from numbers 0-9.

>>> print k
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99]
>>> r= [tuple(k[i:i+2]) for i in xrange(10) if not i%2]
>>> print r
[(0, 1), (2, 3), (4, 5), (6, 7), (8, 9)]
Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131