Python: How to get the length of itertools _grouper

Question

I'm working with Python itertools and using groupby to sort a bunch of pairs by the last element. I've gotten it to sort and I can iterate through the groups just fine, but I would really love to be able to get the length of each group without having to iterate through each one, incrementing a counter.

The project is cluster some data points. I'm working with pairs of (numpy.array, int) where the numpy array is a data point and the integer is a cluster label

Here's my relevant code:

data = sorted(data, key=lambda (point, cluster):cluster)
for cluster,clusterList in itertools.groupby(data, key=lambda (point, cluster):cluster):
    if len(clusterList) < minLen:

On the last line: if len(clusterList) < minLen:, I get an error that

object of type 'itertools._grouper' has no len()

I've looked up the operations available for _groupers, but can't find anything that seems to provide the length of a group.

There may be better ways, but you can convert the iterable to a list and count the elements (`if len(list(clusterList)) < minLen:`). — RocketDonkey, Dec 14 '12 at 00:35
No problem at all - the answers below suggest the same fix, so you can accept one of them to make it 'the answer'. Good luck with everything! — RocketDonkey, Dec 14 '12 at 01:00

kindall · Accepted Answer · 2013-07-31T19:12:26.463

82

Just because you call it clusterList doesn't make it a list! It's basically a lazy iterator, returning each item as it's needed. You can convert it to a list like this, though:

clusterList = list(clusterList)

Or do that and get its length in one step:

length = len(list(clusterList))

If you don't want to take up the memory of making it a list, you can do this instead:

length = sum(1 for x in clusterList)

Be aware that the original iterator will be consumed entirely by either converting it to a list or using the sum() formulation.

edited Jul 31 '13 at 19:12

answered Dec 14 '12 at 00:35

kindall

178,883
35
278
309

2

Such an elegant solution with sum! – egvo Mar 01 '19 at 12:23
`np.bool8(1)` (25 Byte) is smaller than `1` (28 Byte) on my system. – Suuuehgi Aug 28 '21 at 14:06

score 3 · Answer 2 · answered Dec 14 '12 at 00:36

3

clusterList is iterable but it is not a list. This can be a little confusing sometimes. You can do a for loop over clusterList but you can't do other list things over it (slice, len, etc).

Fix: assign the result of list(clusterList) to clusterList.

answered Dec 14 '12 at 00:36

Brian Cain

14,403
3
50
88

score 0 · Answer 3 · answered Mar 01 '21 at 08:16

You can use cardinality package for that. Method count() counts the number of items that iterable yields.

cardinality: determine and check the size of any iterable

The following code gives you the length of clusterList

import cardinality
cardinality.count(clusterList)

Python: How to get the length of itertools _grouper

3 Answers3