6

I'm new to Python and is confused by a piece of code in Python's official documentation.

unique_words = set(word  for line in page  for word in line.split())

To me, it looks equivalent to:

unique_words=set()
for word in line.split():
    for line in page:
        unique_words.add(word)

How can line be used in the first loop before it's defined in the nested loop? However, it actually works. I think it suggests the order of nested list comprehension and generator expression is from left to right, which contradicts with my previous understanding.

Can anyone clarify the correct order for me?

Loopz
  • 199
  • 1
  • 7
  • 1
    You've got the loops backwards. The `for line in page` part should be the outer loop. – APerson Nov 05 '14 at 14:15
  • If you think your nested loop is equivalent, you need to explain where `line` in the outer loop is defined. The order in a nested generator expression is the same as any nested loop. – chepner Nov 05 '14 at 14:21
  • 1
    related : http://stackoverflow.com/q/19484705/674039 – wim Nov 05 '14 at 14:30

6 Answers6

7

word for line in page for word in line.split()

this part works like this:-

for line in page:
    for word in line.split():
        print word

() this makes it `generator function hence overall statement work lie this:-

def solve():
    for line in page:
        for word in line.split():
            yield word

and set() is used to avoid duplicacy or repetition of same word as the code is meant to get 'unique words'.

ni8mr
  • 1,725
  • 3
  • 31
  • 58
Vishnu Upadhyay
  • 5,043
  • 1
  • 13
  • 24
2

From the tutorial in the official documentation:

A list comprehension consists of brackets containing an expression followed by a for clause, then zero or more for or if clauses. The result will be a new list resulting from evaluating the expression in the context of the for and if clauses which follow it. For example, this listcomp combines the elements of two lists if they are not equal: 
>>> [(x, y) for x in [1,2,3] for y in [3,1,4] if x != y]
[(1, 3), (1, 4), (2, 3), (2, 1), (2, 4), (3, 1), (3, 4)]
and it’s equivalent to: 
>>> combs = []
>>> for x in [1,2,3]:
...     for y in [3,1,4]:
...         if x != y:
...             combs.append((x, y))
...
>>> combs
[(1, 3), (1, 4), (2, 3), (2, 1), (2, 4), (3, 1), (3, 4)]
Note how the order of the for and if statements is the same in both these snippets.

See the last sentence quoted above.

Also note that the construct you're describing is not (officially) called a "nested list comprehension". A nested list comprehension entails a list comprehension which is within another list comprehension, such as (again from the tutorial):

[[row[i] for row in matrix] for i in range(4)]

The thing you're asking about is simply a list comprehension with multiple for clauses.

John Y
  • 14,123
  • 2
  • 48
  • 72
  • Note that the first example doesn't answer the question : `x` and `y` are independent and can be swapped, which isn't the case in the OP's example. – Eric Duminil Mar 04 '17 at 22:35
  • 1
    @EricDuminil - it does answer the question. OP wanted to know the correct order for parsing the multiple `for` clauses in a comprehension. Actually, OP already deduced the correct order from observing the behavior, but wanted confirmation. What better confirmation than official documentation? Whether `x` and `y` are independent is irrelevant. The relevant part is unrolling the comprehension to its equivalent nested-loop form, which is incidentally precisely what the accepted and top-voted answer does (except that answer doesn't cite any references to justify it). – John Y Mar 06 '17 at 15:15
  • I still think it's a poor choice of an example (in the documentation, not your answer) because `x` and `y` could be swapped. It doesn't really cover the OP's case where word is in line and line is in page. – Eric Duminil Mar 06 '17 at 15:18
  • @EricDuminil - I understand what you are saying, but the point is that it *does* cover the OP's case, because what matters is the order of the loops. Notice that while `x` and `y` are independent, they are not equal. So if you swap them, you get different results. You seem to be saying that in OP's example, getting the order wrong *breaks* the program. Sure, but so does getting it wrong in the tutorial example if it happens to be in a program which can't handle tuples whose leftmost element is `4`. If you understand the tutorial example, you understand how to parse OP's code snippet. – John Y Mar 06 '17 at 15:43
1

You got the loops wrong. Use this:

unique_words = set(word for line in page for word in line.split())
print unique_words

l = []
for line in page:
    for word in line.split():
        l.append(word)
print set(l)

output:

C:\...>python test.py
set(['sdaf', 'sadfa', 'sfsf', 'fsdf', 'fa', 'sdf', 'asd', 'asdf'])
set(['sdaf', 'sadfa', 'sfsf', 'fsdf', 'fa', 'sdf', 'asd', 'asdf'])
Vincent Beltman
  • 2,064
  • 13
  • 27
0

You have the nested loops mixed. What the code does is:

unique_words={}
for line in page:
    for word in line.split():
        unique_words.add(word)
Yair Daon
  • 1,043
  • 2
  • 15
  • 27
0
for outer_val in outer_loop :
    for inner_val in inner_loop:
        do_something()

Translates to [do_something() for inner_val in inner_loop for outer_val in outer_loop ]

[ op <inner_loop> <outer_loop>]

Ahmad Shapiro
  • 1
-2

In addition to the right answers that stressed the point of the order, I would add the fact that we use set to delete duplicates from line to make "unique words". check this and this thread

unique_words = set(word for line in page for word in line.split())
print unique_words

l = {}
for line in page:
    for word in line.split():
        l.add(word)
print l
Community
  • 1
  • 1
user3378649
  • 5,154
  • 14
  • 52
  • 76