3

I come from a Java background and just started to work on Python. Most of the things are fairly easy to pick up but I am having hard time to understand one thing in the language which I just found out that is called list comprehension. What is this list comprehension in Python? How does this compare with language constructs found in Java? The problem is it's everywhere, nearly all the examples I found here and there use it.

For the following example, allow me to understand how this works.

[x**2 for x in range(10)]

And then there this.

[j + k for j in 'abc' for k in 'def']

Beyond that I also have seen things like this somewhere on Stackoverflow.

(x for x in (0,1,2,3,4))

Also things like this.

total = sum(x+y for x in (0,1,2,3) for y in (0,1,2,3) if x < y)

This started to get messy, could you please help me?

shuttle87
  • 15,466
  • 11
  • 77
  • 106
  • 2
    Have you tried to run these in your python shell? List comprehension are a way to generate lists. – vmonteco Sep 06 '15 at 16:35
  • What exactly are you confused about? When you have this type of expression within `[]` it is a "list comprehension", within `{}` is a "dict comprehension" and within `()` is a "generator expression", but all of them work similarly, it's just that the type at the end differs. – Cory Kramer Sep 06 '15 at 16:36
  • There's 2 parts to this, some rule to generate a sequence and a data type to store that generated sequence in. First you must have a rule to generate a sequence of values, for example `x**2 for x in range(10)` iterates over `range(10)` by setting `x` to each value in that sequence and then generates the sequence of `x**2` for each value. Because this is of the form `[sequence]` you get a list. If it were `{sequence}` you would get a set, of if it was `(sequence)` you would get a generator expression. – shuttle87 Sep 06 '15 at 16:39
  • yes I have tried these and trying to understand the difference, as there are lots of it and I get confused in each of them :( –  Sep 06 '15 at 16:41
  • 3
    It might be easy for you to say if you are not starting a new language but I already checked other answers. I truly want to understand this :( –  Sep 06 '15 at 16:49
  • Although these questions are closely related, I am not sure that this is a duplicate. – Semih Yagcioglu Sep 06 '15 at 16:57
  • 4
    This is a bit annoying, I wrote a long answer explaining how this works from a Java perspective but it got closed before I could post. Python and Java have a very different approach to this type of problem. – shuttle87 Sep 06 '15 at 17:03
  • 2
    Can't agree more. We should be more welcoming otherwise #SOreadytohelp tag wouldn't mean much right? – Semih Yagcioglu Sep 06 '15 at 17:06
  • I ended up writing a [tutorial](http://www.jaggedverge.com/2015/10/python-comprehension-syntax-for-java-developers/) about this. – shuttle87 Nov 19 '15 at 22:06

1 Answers1

6

What is this list comprehension in Python?

First let’s start with the basic definition taken from the official Python documentation.

A list comprehension consists of brackets containing an expression followed by a for clause, then zero or more for or if clauses. The result will be a new list resulting from evaluating the expression in the context of the for and if clauses which follow it.

The problem is it's everywhere, nearly all the examples I found here and there use it.

List comprehension is a very flexible concept. It allows us to define lists as we know from the mathematics. Say we have a set S in which each element of S is a square of a number x and x could only takes values ranging from 0 to 10.

See the above definition. It took a paragraph to describe it. But there is a better way to describe it.

S = {x² : x in {0 ... 10}}

That’s why I love math, it is always to the point. Now remember your first example?

S = [x**2 for x in range(10)]

That’s the set we just defined. Neat right? That’s why it is used so much. (Don’t get confused with the x**2 syntax, because Python follows a weird syntax here which you probably might be familiar as x^2 from other languages.)

In Python you can iterate over pretty much anything. Which basically means you can easily iterate over each character of a string. Let’s look at the second example. It just iterates over the words ‘abc’ and ‘def’ and creates a list out of them.

lst = [j + k for j in 'abc' for k in 'def']

Notice that we assigned this to a list named lst. It was no coincidence. Python is the most humane programming language I have ever laid eyes on. So she will help you when you get stuck. Like this.

help(lst)

You can now see what you can do with lst. Ever got confused what lst is? You can check what it is via type.

print type(lst)

Before we move forward let’s talk a little bit about iterators and generators.

Iterators are objects that you can call next() method on. Like this.

iterator = iter([1,2,3,4])

Now we can print the first element of our iterator, like this.

print iterator.next()

Now we can talk about the generators. They are functions that generate iterators. There is however one other concept called generator expressions.

(x for x in (0,1,2,3,4))

This is a generator expression. A generator expressions is like a shortcut to build generators out of expressions similar to that of list comprehensions.

total = sum(x+y for x in (0,1,2,3) for y in (0,1,2,3) if x < y)

What above line does is to first create a generator using a generator expression and iterate over each element and sum those elements.

Corey Goldberg
  • 59,062
  • 28
  • 129
  • 143
Semih Yagcioglu
  • 4,011
  • 1
  • 26
  • 43
  • OMG. Thank you very much! –  Sep 06 '15 at 17:00
  • Also note that the "weird syntax" Python uses here is because of `^` being a bitwise operator. In fact, it is such in many languages. `**` is just a convenient operator for `math.pow(x, y)`. – Two-Bit Alchemist Sep 10 '15 at 18:23
  • Thanks for noting that, this is definetly a good remark for future references to this question. – Semih Yagcioglu Sep 10 '15 at 18:25
  • I was actually trying to look into what other languages do here. I know C/C++/Java don't have an operator at all, only a function equivalent to Python's `pow`. [Some languages](http://rosettacode.org/wiki/Exponentiation_operator) apparently provide `^^` (same reason; to distinguish bitwise `^`) and only BASIC seems to exponentiate with `^`. – Two-Bit Alchemist Sep 10 '15 at 18:28
  • Wow, I thought it was more common, but if that's the case, I think I should remove that statement. I don't want to mislead anyone :) – Semih Yagcioglu Sep 10 '15 at 18:32