Dynamically add properties to iterator in Python 2

Question

I'm trying to add to an iterator, some custom properties like:

iterator = iter([1, 2, 3, 4, 5, 6, 7, 8, 9, 0])
iterator.__custom_property__ = 'random value'

But all I get is this error that says that the iterator attribute are read only:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'listiterator' object has no attribute '__custom_property__'

This question is linked to this other one: Get length of a (non infinite) iterator inside it's loop using Python 2.7.

What I thought is this: If I can somehow add some custom data into the iterator object that the itertools.imap returns to me, I could access to those information inside any loop that loops over the iterator.

But, apparently, those data cannot be stored.

At least using this workflow.

Does anyone know how to "hack" into an iterator object (specifically a listiterator) and define some custom attributes or methods?

Thanks!

INFO: I thought that, because of the argument of this question, could be worth asking in another thread instead of mixing the information in the other one.

What's the use case here? If you're trying to assign something from outside of the iterator then you already know what it is so can access it anyway... so, a little bit puzzled what the use is here? I mean - if you really want, why can't you just write your own class that behavers like an iterator but provides what you want on top? — Jon Clements, Oct 30 '17 at 16:51
Basically I have a database query that gives me some information. Those information are processed through an iterator (that I cannot change) and the function returns this iterator. What I have to do is to get the number of rows returned from the query, but I need this information into the for loop that iterate over this iterator. And I thought that if I could somehow store this information into the iterator object, I could end up having an iterator that knows it's own size. — Andrea Rastelli, Oct 30 '17 at 16:53
I'm confused too. All properties of an object in Python are publicly accessible. Are you trying to make a convenience method to access some deeply nested property? — Ananth Rao, Oct 30 '17 at 16:53
@AndreaRastelli a guaranteed way is to use `list(some_iterable)` and then take its length... Depending how the DB/library you're using works - it may not know what the resulting length is if it's streaming data or returning data in pages and only knows when it's done by there being no more data... (ergo you have to either exhaust it first or find the length apriori...) — Jon Clements, Oct 30 '17 at 16:56
@JonClements I thought that some custom information could be stored inside an iterator to be passed over in the flow of my application. In this specific use-case I simply wanted to use the best of the iterator (so no need to loop over all the elements of the object I have) and the best of the db (the size of the resulting query) together. — Andrea Rastelli, Oct 30 '17 at 17:03
@Andrea iterators are meant to be very, very simple "get next item or signal when empty" objects... Anything else you need to do as user2357112 has shown you have to make your own version. — Jon Clements, Oct 30 '17 at 17:06
@JonClements Yeah, I was trying to design something that behave like user2357112 suggested — Andrea Rastelli, Oct 30 '17 at 17:09
@AndreaRastelli but *why on the iterator*? You can always just store the size in some local variable, no? Or put your iterator into a tuple, `(it, it_size)` or something... In other words, I don't understand why "easy to be accessed into the for-loop that iterates over the iterator information" translates into "as an attribute on the iterator"? How are you getting the size, is it returned, along with the iterator, from some function? — juanpa.arrivillaga, Oct 30 '17 at 17:11
Well.. I have to change something in an existing code, and because this function returns an iterator, and could be used somewhere else in the code base, I didn't want to change all existing occurrence of the current function. This is why extending the iterator behavior seemed to be a good idea at first. — Andrea Rastelli, Oct 30 '17 at 17:31

score 2 · Answer 1 · answered Oct 30 '17 at 16:54

You can't. The iterator in your example simply doesn't have anywhere to put more attributes; it doesn't have a __dict__. The closest you can do is wrap it in another iterator that does support extra attributes:

class ProxyIter(object):
    def __init__(self, wrapped):
        self.wrapped = wrapped
    def __iter__(self):
        return self
    def __next__(self):
        return next(self.wrapped)
    next = __next__

iterator = ProxyIter(iter([1, 2, 3, 4, 5, 6, 7, 8, 9, 0]))
iterator.foo = 3

though I'd question whether sticking extra attributes on an iterator is really the best way to provide access to this data.

Well, as I was saying, I have to access to the size of the query that my iterator iterates into. I can get this info easly, but I need to store this info somewere, easy to be accessed into the for-loop that iterates over the iterator information. And because the iterator I'm working on is always a non-infinite one, I thought that could be useful to have an iterator aware of it's size. But apparently I need to create some kind of wrapper to do that.. — Andrea Rastelli, Oct 30 '17 at 16:58

Dynamically add properties to iterator in Python 2

1 Answers1