Common errors and pitfalls
Trying to append elements by assigning to a missing index
Sometimes people mistakenly try to implement the loop code with something like:
xs = [1, 3, 5]
ys = []
for i, x in enumerate(xs):
ys[i] = x + 1
It is only possible to assign to indices in a list that are already present - but here, the list starts out empty, so there is nothing present yet. The first time through the loop will raise an IndexError. Instead, use the .append method to append the value.
There are other, more obscure ways, but there is no real point to them. In particular: "pre-allocating" the list (with something like ys = [None] * len(xs) may offer a small performance improvement in some cases, but it's ugly, more error prone, and only works if the number of elements can be known ahead of time (e.g., it won't work if xs actually comes from reading a file using the same loop).
Using append incorrectly
The append method of lists returns None rather than the list that was appended to. Sometimes people mistakenly try code like:
xs = [1, 3, 5]
ys = []
for x in xs:
ys = ys.append(x) # broken!
The first time through the loop, ys.append(x) will modify the ys list, and evaluate to None, and then ys = will assign that None to ys. The second time through, ys is None, so the call to .append raises an AttributeError.
list.append in a comprehension
Code like this will not work:
# broken!
xs = [1, 3, 5]
y = []
y = [y.append(x + 1) for x in xs]
Sometimes this results from unclear thinking; sometimes it results from attempting to convert old code with a loop to use a comprehension, and not making all the necessary changes.
When done deliberately, it shows a misunderstanding of the list comprehension. The .append method returns None, so that is the value that ends up (repeatedly) in the list created by the comprehension. But more than that, it's conceptually wrong: the purpose of the comprehension is to build the list from the calculated values, so calling .append makes no sense - it's trying to do work that the comprehension is already responsible for. Although it's possible to skip the assignment here (and then y has already had the appropriate values appended), it is poor style to use a list comprehension for its side effects - and especially so when those side effects do something the comprehension could do naturally.
Re-creating a new list inside the loop
The key point in the explicit loop code is that ys is set to an initial empty or list or dictionary once. It does need to happen (so that elements can be added or keys can be inserted), but doing it inside the loop means that the result will keep getting overwritten.
That is, this code is broken:
def broken_list_with_inline_code_and_for():
for x in [1, 3, 5]:
ys = []
ys.append(x + 1)
return ys
This should be obvious once it's explained, but it's a very common logical error for new programmers. Each time through the loop, ys becomes [] again, and then one element is added - before becoming [] again, the next time through the loop.
Sometimes people do this because they think that ys should be "scoped to" the loop - but this isn't good reasoning (after all, the entire point is to be able to use ys after the loop has completed!), and anyway Python does not create separate scopes for loops.
Trying to use multiple inputs without zip
Code using a loop or a comprehension needs special handling in order to "pair up" elements from multiple input sources. These ways will not work:
# broken!
odds = [1, 3, 5]
evens = [2, 4, 6]
numbers = []
for odd, even in odds, evens:
numbers.append(odd * even)
# also broken!
numbers = [odd * even for odd, even in odds, evens]
These attempts will raise a ValueError. The problem is that odds, evens creates a single tuple of lists; the loop or comprehension will try to iterate over that tuple (so the value will be [1, 3, 5] the first time through and [2, 4, 6] the second time through), and then unpack that value into the odd and even variables. Since [1, 3, 5] has three values in it, and odd and even are only two separate variables, this fails. Even if it did work (for example, if odds and evens were coincidentally the right length), the results would be wrong, since the iteration is in the wrong order.
The solution is to use zip, like so:
# broken!
odds = [1, 3, 5]
evens = [2, 4, 6]
numbers = []
for odd, even in zip(odds, evens):
numbers.append(odd * even)
# or
numbers = [odd * even for odd, even in zip(odds, evens)]
This is not a problem when using map instead of the loop or comprehension - the pairing-up is done by map automatically:
numbers = list(map(lambda x, y: x * y, odds, evens))
Trying to modify an input list
List comprehensions create a new list from the input, and a map similarly iterates over the new results. Neither of these is appropriate for trying to modify the input list directly. However, it is possible to replace the original list with the new one:
xs = [1, 3, 5]
ys = xs # another name for that list
xs = [x + 1 for x in xs] # ys will be unchanged
Or replace its contents using slice assignment:
xs = [1, 3, 5]
ys = xs
# The actual list object is modified, so ys is changed too
xs[:] = [x + 1 for x in xs]
Given an input list, an explicit loop can be used to replace the list elements with the results from the calculation - however, it's not straightforward. For example:
numbers = [1, 2, 3]
for n in numbers:
n += 1
assert numbers == [1, 2, 3] # the list will not change!
This kind of list modification is only possible if the underlying objects are actually modified - for example, if we have a list of lists, and modify each one:
lol = [[1], [3]]
for l in lol:
# the append method modifies the existing list object.
l.append(l[0] + 1)
assert lol == [[1, 2], [3, 4]]
Another way is to retain the index and assign back to the original list:
numbers = [1, 2, 3]
for i, n in enumerate(numbers):
numbers[i] = n + 1
assert numbers == [2, 3, 4]
However, in almost every normal circumstance it will be a better idea to create a new list.
A not-so-special case: lowercasing a list of strings
Many duplicates of this question specifically seek to convert an input list of strings all to lowercase (or all to uppercase). This is not special; any practical approach to the problem will involve solving the problems "lowercase a single string" and "repeat a calculation and collect the results" (i.e. this question). However, it is a useful demonstration case because the calculation involves using a method of the list elements.
The general approaches look like this:
def lowercase_with_explicit_loop(strings):
result = []
for s in strings:
result.append(s.lower())
return result
def lowercase_with_comprehension(strings):
return [s.lower() for s in strings]
def lowercase_with_map(strings):
return list(map(str.lower, strings))
However, there are two interesting points to make here.
Note how the map version differs. Although it is of course possible to make a function that takes in a string and returns the result of the method call, it is not necessary. Instead, we can directly look up the lower method from the class (here, str), which in 3.x results in a perfectly ordinary function (and in 2.x results in an "unbound" method which can then be called with the instance as an explicit parameter - which amounts to the same thing). When a string is passed to str.lower, then, the result is a new string which is the lowercase version of the input string - i.e., exactly the function needed for map to work.
The other approaches do not allow this kind of simplification; looping or using a comprehension / generator expression requires choosing a name (s in these examples) for the iteration (loop) variable.
Sometimes, when writing the explicit loop version, people expect to be able to just write s.lower() and thus transform the string in-place, within the original strings list. As stated above, it is possible to modify lists with this general sort of approach - but only with methods that actually modify the object. Python's strings are immutable, so this doesn't work.
