8

I like the Python syntax a lot, but as I'm coming from C++ I don't get one thing about iterators in Python. In C++ there are 2 kinds of iterators - constant and modifying (non-const). In python it seems (for what I've seen) like there is only the first kind and if you want to modify the elements, you have to use the indexing, which doesn't feel comfortable and so generic to me. Let me illustrate with a simple example:

ab = ["a", "b"]
for (index, lett) in enumerate(ab):
    print "Firstly, lett is ab[index]?", lett is ab[index]
    lett = str(index)
    print lett
    print ab[index]
    print "But after, lett is ab[index]?", lett is ab[index]

So I wasn't able to modify the list with the iterator. It just makes Lazy copies (see Wikipedia) as I discovered with the is operator, so is there a way to say I want it to be a directly modifying iterator instead using the neat

for variable in iterable_object:

syntax?

Community
  • 1
  • 1
Huge
  • 661
  • 7
  • 14

4 Answers4

9

The syntax

for x in iterable

does not create any lazy copies -- it assigns the exact objects in the list to x one after the other. If these objects are mutable, you can modify them:

a = [[1, 2], [3, 4]]
for x in a:
    x.append(5)
print a

prints

[[1, 2, 5], [3, 4, 5]]

Your example uses a list of strings. Strings are immutable in Python, so you can't modify them.

Sven Marnach
  • 574,206
  • 118
  • 941
  • 841
  • Well, there are some insights in your post, but this is not what I meant. Probably I should have used the word change or replace instead of modify, because I'd like to change the whole element. You can see it in my example code. I now it can change the type of the element, but as I know, they are all `object' so it shouldn't matter. – Huge Jun 23 '11 at 14:56
3
def iset(iterable):
    for i, x in enumerate(iterable):
        def setter(value):
            iterable[i] = value
        yield setter, x
a = range(10)
for set_x, x in iset(a):
    set_x(x * 2)
print a

prints

[0, 2, 4, 6, 8, 10, 12, 14, 16, 18]
phant0m
  • 16,595
  • 5
  • 50
  • 82
  • 1
    That is more what I wanted, but I'm afraid not every iterable must have this [] operator, consider a graph (which is not a part of the python standard library), what meaning would [] have on it's objects? Also your solution is not very straightforward, though it's near to what I'd wish. – Huge Jun 23 '11 at 15:02
  • 1
    The problem is, that what you want is not really possible. Any object can define an iterator and have it behave however it pleases. As a consequence, there is no universal way to modify the value (or rather: replace) an immutable object that you're being returned from the iterator in its source - if there even is such a "source". (The "iterator" could just generate values if you (a native example: generators)). This is why every such workaround must be specifically designed to match the iterable you're working with in one way or another. ofc, this restricts how you can swap your objects. – phant0m Jun 23 '11 at 15:23
  • I see, I just supposed that there is something as element iterators, which would return some 'reference' to the object, so it could be swapped. Thanks for clarification anyway. – Huge Jun 23 '11 at 21:31
  • In Python, only identifiers/names (variables) are references to objects. You cannot have an identifier always refer to what a different identifiers refers to (think: Pointer). Rather, two identifiers can point to the very same object, but they identifiers themselves are never connected. – phant0m Jun 23 '11 at 21:36
  • Well, as I see it, the identifiers are 'connected' by the object (data), they refer to, like when you write *ptr = new_value, then both of the pointer get the new value (the old being rewritten). And this data replacing (after dereference) is what I'm missing in Python. – Huge Jun 28 '11 at 12:48
  • No, they do not both get the new value. Rather, they just happen to point to the same location that was being written to. If they were connected, you could think of them like this: `ptr = &value; ptr_ptr = &ptr;` Now `*ptr` and `**ptr_ptr` will always be the same. If I change `ptr` to point to somewhere else, `*ptr == **ptr_ptr` will still hold true. In Python, you never get such connections. The variables can point to the very same object. As soon as you assign something to one of those variables, it will no longer be the case. – phant0m Jun 30 '11 at 09:06
0

The issue here is not in fact a problem with how iterators work in python as opposed to how assignment works on most python types. When the asker attempts to overwrite the value of lett which is indeed an alias to ab[index] and should seem logically to work, this is not in fact what is happening, instead lett (the reference or pointer lett, not the value it points to) is being reassigned to point to a constant of it's new value, this is quite different from overwriting the bytes at the memory location it points to. This way of working is necessary to allow python's duck typing to work where a variable name can over time point to different types with different sizes. Please see this page for more explanation of what is happening here: http://gestaltrevision.be/wiki/python/aliases

The closest we could achieve is a hand made 'mutable integer' type which will allow it's underlying value to change unlike python ints. However there is little point in going over this by hand here as it has already been explained in this question. Though the question is quite different it is the same underlying issue and the solution is equally acceptable. However if you are doing this I would consider first if you can restructure your code to avoid this in the first place as it is quite a dangerous and accident prone way of working.

Here's an example from an answer to the "increment int object" question for a full explanation of how to hack this together please see the question linked above, note that this example only includes decrementing, the other operators would have to be implemented in the same way:

import sys
class FakeInt(int):
    def __init__(self, *arg, **kwarg):
        self._decr = False
        int.__init__(self, *arg, **kwarg)
    def __neg__(self):
        if self._decr:
            upLocals = sys._getframe(1).f_locals
            keys, values = zip(*upLocals.items())
            i = list(values).index(self)
            result = FakeInt(self-1)
            upLocals[keys[i]]=result
            return result
        self._decr = not self._decr
        return self
Community
  • 1
  • 1
Vality
  • 6,577
  • 3
  • 27
  • 48
0

A similar scenario...

>>> a = b = 0
>>> a = 42
>>> a, b
(42, 0)

>>> a = b = [0]
>>> a[0] = 42
>>> a, b
([42], [42])

While python uses references internally, and both a and b point to the same 0 object, saying a = 42 replaces a, and not the thing a references. A list can be used as a workaround, but it's by no means elegant.

The iterator is just like a, it is the actual thing but there's no way to "dereference", and I can imagine adding this feature would break a whole bunch of other things.

I think the enumerate method is still the right way to go.

jozxyqk
  • 16,424
  • 12
  • 91
  • 180