1

I'm working with some iterator generated using itertools.imap, and I was thinking if there is a way to access the iterator length inside the for-loop that I use to loop over the elements.

What I can say for sure is that the iterator doesn't generate an infinite amount of data.

Also, because the information I'm looping are from a query to a database, I can get the length of the information from there, but the function I'm using has to return an iterator.

I thought of some options:

def iterator_function(some, arguments, that, I, need):
    query_result = query()
    return_iterator = itertools.imap(
        mapping_function,
        query_result
    )
    return return_iterator

Because I cannot change the returned iterator, I thought of something (really ugly) like:

query_result = query()
query_size = len(query_result)

return_iterator = itertools.imap(
    lambda item: (mapping_function(item), query_size),
    query_result
)

return return_iterator

But I don't really like this option, and I was thinking if there is a way, in Python, to get the iterator size from inside the loop over the iterator, something like:

for item in iterator():
    print item.iterator_length()
    # do other stuff

Or even something like:

for item in iterator():
    print iterator.length() # or iterator().length()???

Thanks!

Andrea Rastelli
  • 617
  • 2
  • 11
  • 26
  • https://stackoverflow.com/questions/3345785/getting-number-of-elements-in-an-iterator-in-python might help – uspectaculum Oct 30 '17 at 13:02
  • Yeah, I saw that question, but they are basically saying that I have to convert my iterator into a list and get the list length.. but at that point I have lost all the usefullness of having an Iterator.. right? I mean, if I have a query with LOTS of results, I would like to get the length of the iterator without reading all the row of the result.. – Andrea Rastelli Oct 30 '17 at 13:05
  • You can use the iterator twice. First to count the length, then to use the elements. – MegaIng Oct 30 '17 at 13:07
  • @MegaIng how can you use an iterator twice? – Aleksandr Borisov Oct 30 '17 at 13:08
  • Yes, but there is no way to find the length of an iterator as you can't simply find the length of something that hasn't been created yet. Hence why people prefer to call `list(...)` over it and find the length, because that generates all the iterations until the end and puts it into a neat list. – uspectaculum Oct 30 '17 at 13:10
  • @клйкбаутоьмажазвайкукас I'm ok with the concept "you cannot find the length of something that hasn't been created yet". What I was curious to understand is if there is a way to ask the iterator the number of elements it's going to iterate when such elements are finite (i'm ok returning an exception for all other situations). But I guess that there is no such a solution, and I may need to design one for my needs. – Andrea Rastelli Oct 30 '17 at 15:41
  • Oh, well then yes- if you're simply finding the maximum value at which a `StopIteration` exception is raised then you simply need to find the underlying value which upon reach, causes the `__next__` model to break. E.g. `if current_iteration_count > self.maximum_count`, you're looking for the `self.maximum_count`. – uspectaculum Oct 30 '17 at 16:01

2 Answers2

0

I don't know if my idea is correct but how about class generator pattern and trying to add sequence bahaviour :

if your class represents something that has a length, don't define a GetLength method; define the __len__ method and use len(instance).

something like this :

class firstn(object):
    def __init__(self, n):
        self.n = n
        self.num, self.nums = 0, []

    def __iter__(self):
        return self

    # Python 3 compatibility
    def __next__(self):
        return self.next()

    #    V------- Something like this 
    def __len__(self):
        return self.my_length

    def next(self):
        if self.num < self.n:
            cur, self.num = self.num, self.num+1
            return cur
        else:
            raise StopIteration()
Take_Care_
  • 1,957
  • 1
  • 22
  • 24
  • About this solution, I'm not sure if this matches what I need, because the code I'm trying to change uses itertools.imap, so I cannot hack into the object returned by this function. And I cannot see the point in creating an iterable object that returns the length of my iterator calculating `len(list(iterator))` – Andrea Rastelli Oct 30 '17 at 15:36
0

Also, because the information I'm looping are from a query to a database, I can get the length of the information from there, but the function I'm using has to return an iterator.

Assuming you have a good database, do the count there. Doubt any solution in python will be faster/cleaner.

The Pjot
  • 1,801
  • 1
  • 12
  • 20
  • What I meant was to have a way to get the information that I need using some property of the iterable, from inside the loop.. I can define the length of the query result almost everywere, and I'm pretty much ok with that, I was just curious if Python allows you to have the length of an iterator that iterate over a finite set of objects. – Andrea Rastelli Oct 30 '17 at 15:38
  • 1
    I see some iterators have a '__length_hint__()' function.. which seems to return the 'remaining' items length. – The Pjot Oct 30 '17 at 15:56