8

I recently started learning Python, and the concept of for loops is still a little confusing for me. I understand that it generally follows the format for x in y, where y is just some list.

The for-each loop for (int n: someArray) becomes for n in someArray,

And the for loop for (i = 0; i < 9; i-=2) can be represented by for i in range(0, 9, -2)

Suppose instead of a constant increment, I wanted i*=2, or even i*=i. Is this possible, or would I have to use a while loop instead?

martineau
  • 119,623
  • 25
  • 170
  • 301
user1320925
  • 117
  • 1
  • 3

5 Answers5

12

As you say, a for loop iterates through the elements of a list. The list can contain anything you like, so you can construct a list beforehand that contains each step.

A for loop can also iterate over a "generator", which is a small piece of code instead of an actual list. In Python, range() is actually a generator (in Python 2.x though, range() returned a list while xrange() was the generator).

For example:

def doubler(x):
    while True:
        yield x
        x *= 2

for i in doubler(1):
    print i

The above for loop will print

1
2
4
8

and so on, until you press Ctrl+C.

Greg Hewgill
  • 951,095
  • 183
  • 1,149
  • 1,285
  • I would say a standalone generator is overkill for this - a generator expression would probably do it just as well. E.g: `for i in (x*2 for x in range(10)):` – Gareth Latty May 03 '12 at 23:08
  • 1
    Yes, you can write the code more compactly. However, I think it is instructive to show a taste of what a generalised generator can do. – Greg Hewgill May 03 '12 at 23:09
  • While it's nice to know you can do more with it, it's not good to advise people to overcomplicate things. Explaining the generator expression syntax as well would probably be good, to avoid sending the asker off making very simple generators as full functions. – Gareth Latty May 03 '12 at 23:10
  • I encourage you to add an answer demonstrating the generator expression syntax. – Greg Hewgill May 03 '12 at 23:11
  • Done, and +1 for yours - it's a good answer, one can do magic with generators. – Gareth Latty May 03 '12 at 23:23
8

You can use a generator expression to do this efficiently and with little excess code:

for i in (2**x for x in range(10)): #In Python 2.x, use `xrange()`.
    ...

Generator expressions work just like defining a manual generator (as in Greg Hewgill's answer), with a syntax similar to a list comprehension. They are evaluated lazily - meaning that they don't generate a list at the start of the operation, which can cause much better performance on large iterables.

So this generator works by waiting until it is asked for a value, then asking range(10) for a value, doubling that value, and passing it back to the for loop. It does this repeatedly until the range() generator yields no more values.

Community
  • 1
  • 1
Gareth Latty
  • 86,389
  • 17
  • 178
  • 183
  • 2
    @TomWijsman I don't think it's any less readable. If you really wanted you could do `doubles = (x*2 for x in range(10))` and then loop over `doubles`. If you are comparing it to your answer - a list comprehension - we are literally talking about different brackets outside the expression. How is that less readable or maintainable? – Gareth Latty May 03 '12 at 23:26
  • 2
    @TomWijsman My answer is a one liner - and to suggest that a single line is always better than multiple lines is insane. Sometimes more lines are more readable. As to debugging being harder, that simply isn't true. As I said in my other comment - there is a reason why all of Python 3's builtins are now lazy where in 2.x they produced lists. – Gareth Latty May 03 '12 at 23:35
  • 1
    @Lattyware I think the OP wants 2,4,8,16,32... and your solution gives 2,4,6,8,10.... – Ashwini Chaudhary May 03 '12 at 23:40
  • @AshwiniChaudhary Whoops, fixed. – Gareth Latty May 04 '12 at 00:00
5

Bear in mind that the 'list' part of the Python can be any iterable sequence.

Examples:

A string:

for c in 'abcdefg':
   # deal with the string on a character by character basis...

A file:

with open('somefile','r') as f:
    for line in f:
         # deal with the file line by line

A dictionary:

d={1:'one',2:'two',3:'three'}
for key, value in d.items():
   # deal with the key:value pairs from a dict

A slice of a list:

l=range(100)
for e in l[10:20:2]:
    # ever other element between 10 and 20 in l

etc etc etc etc

So it really is a lot deeper than 'just some list'

As others have stated, just set the iterable to be what you want it to be for your example questions:

 for e in (i*i for i in range(10)):
     # the squares of the sequence 0-9...

 l=[1,5,10,15]
 for i in (i*2 for i in l):
     # the list l as a sequence * 2...
  • +1 - This is a good point. For loops go over any iterable, not just lists. – Gareth Latty May 03 '12 at 23:23
  • I don't see how this answers the question. – Tamara Wijsman May 03 '12 at 23:33
  • @Tom Wijsman: The OP states `the concept of for loops is still a little confusing for me. I understand that it generally follows the format for x in y, where y is just some list.` I was clarifying that y is a lot more than 'just some list' –  May 03 '12 at 23:35
1

You will want to use list comprehensions for this

print [x**2 for x in xrange(10)] # X to the 2nd power.

and

print [x**x for x in xrange(10)] # X to the Xth power.

The list comprehension syntax is a follows:

[EXPRESSION for VARIABLE in ITERABLE if CONDITION]

Under the hood, it acts similar to the map and filter function:

def f(VARIABLE): return EXPRESSION
def c(VARIABLE): return CONDITION

filter(c, map(f, ITERABLE))

Example given:

def square(x): return x**2

print map(square, xrange(10))

and

def hypercube(x): return x**x

print map(hypercube, xrange(10))

Which can be used as alternative approach if you don't like list comprehensions. You could as well use a for loop, but that would step away from being Python idiomatic...

Tamara Wijsman
  • 12,198
  • 8
  • 53
  • 82
  • Creating a list and then looping over it isn't optimal, and that's not the syntax for a list comp (it's an expression, not a function, it's an iterable, not a list, and there could be an if statement at the end). – Gareth Latty May 03 '12 at 23:13
  • @Lattyware: It is optimal, it's what a generator expression does. Note how your answer links to list comprehensions... ;) – Tamara Wijsman May 03 '12 at 23:18
  • Not true. My answer links to a video I made that explains list comprehensions alongside generator expressions and dict and set comprehensions. Generator expressions are lazy, and therefore **very** different to a list comprehension. If we do a loop to five million, your solution will create a list from `0*2` through `4999999*2`, then loop over it. A generator expression will calculate them as needed. – Gareth Latty May 03 '12 at 23:21
  • A list could be what he wants, in which case a list comprehension is best, but as he didn't say so, I think it's premature to presume so. – Gareth Latty May 03 '12 at 23:24
  • @Lattyware: I'll quote Donald Knuth: `Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.` – Tamara Wijsman May 03 '12 at 23:26
  • 3
    That is entirely true, and yet in this case, we are talking about a different set of brackets around an expression. It's not like this optimisation will cost us time, readability or anything else, so why not do it? The quote falls down when we are talking about something that costs us no time or effort, and being in good practice often means that you don't need to worry about it when it does matter. – Gareth Latty May 03 '12 at 23:28
  • You say 'known to be' - that's simply not true. There is a reason why in Python 3.x, they changed all of the built-in operations to lazy evaluation. It is in no way more difficult to debug. – Gareth Latty May 03 '12 at 23:33
  • Feel free to prove yourself right. The burden of proof is on the accuser. In what situation does having a lazily evaluated expression making debugging harder? – Gareth Latty May 03 '12 at 23:36
  • That link isn't relevant. It doesn't talk about debugging being harder, and it's talking about designing it in relation to the language. Python manages lazy evaluation for you with generators. Show me an example of code being less maintainable or debuggable in Python thanks to a generator. – Gareth Latty May 03 '12 at 23:43
  • 1
    I have not downvoted any other answers - in fact, I upvoted two of the others. The reality is that debugging in Python with generator expressions is fine - debugging in Python is almost always done with `print()` statements and so it really doesn't matter. If worst comes to worst you can just wrap your generator in `list()` to see it's output. I have debugged plenty of problems in Python while using generators. As to keeping it simple - a generator expression is no more complex than a list comprehension. – Gareth Latty May 03 '12 at 23:53
  • @Lattyware: As the brackets can be replaced to allow for debugging (supposing the existence of a debugger, whether or not one exists), I agree. I'll revert my down vote *using magic*... – Tamara Wijsman May 03 '12 at 23:55
  • @Lattyware: Well, the nice thing is that our conversation actually [helped to improve Wikipedia](http://en.wikipedia.org/w/index.php?title=Lazy_evaluation&diff=490549486&oldid=487487101) if no people decide to go and roll it back. Sorry and thanks... :) – Tamara Wijsman May 04 '12 at 00:00
  • Nothing wrong with a good discussion - I've been wrong before many a time, and I will be again in the future - I'd rather know about it than not. If I am right, there is nothing wrong with knowing why you think something, and hopefully someone else benefits. – Gareth Latty May 04 '12 at 00:03
  • @Lattyware: Removed the comments that were based on the wrong fact. That's exactly the reason why I don't step away from a good discussion, it improves something in one way or the other. And even just doing it for practicing discussion, is already a good reason. As long as it's kept scientific and not just two straw mans, which is easy to fall into ... – Tamara Wijsman May 04 '12 at 00:06
0

Just for an alternative, how about generalizing the iterate/increment operation to a lambda function so you can do something like this:

for i in seq(1, 9, lambda x: x*2):
    print i
...
1
2
4
8

Where seq is defined below:

#!/bin/python
from timeit import timeit

def seq(a, b, f):
    x = a;
    while x < b:
        yield x
        x = f(x)

def testSeq():
    l = tuple(seq(1, 100000000, lambda x: x*2))
    #print l

def testGen():
    l = tuple((2**x for x in range(27)))
    #print l

testSeq();
testGen();

print "seq", timeit('testSeq()', 'from __main__ import testSeq', number = 1000000)
print "gen", timeit('testGen()', 'from __main__ import testGen', number = 1000000)

The difference in performance isn't that much:

seq 7.98655080795
gen 6.19856786728

[EDIT]

To support reverse iteration and with a default argument...

def seq(a, b, f = None):
    x = a;
    if b > a:
        if f == None:
            f = lambda x: x+1
        while x < b:
            yield x
            x = f(x)
    else:
        if f == None:
            f = lambda x: x-1
        while x > b:
            yield x
            x = f(x)

for i in seq(8, 0, lambda x: x/2):
    print i

Note: This behaves differently to range/xrange in which the direction </> test is chosen by the iterator sign, rather than the difference between start and end values.

jozxyqk
  • 16,424
  • 12
  • 91
  • 180