Python conciseness confuses me

Question

I have been looking at Pandas: run length of NaN holes, and this code fragment from the comments in particular:

Series([len(list(g)) for k, g in groupby(a.isnull()) if k])

As a python newbie, I am very impressed by the conciseness but not sure how to read this. Is it short for something along the lines of

myList = []
for k, g in groupby(a.isnull()) :
    if k:
        myList.append(len(list(g)))
Series(myList)

In order to understand what is going on I was trying to play around with it but get an error:

list object is not callable

so not much luck there.

It would be lovely if someone could shed some light on this.

Thanks, Anne

You just need to learn about Python comprehensions... list comprehensions, set comprehensions, etc. — Charles Salvia, Jul 17 '13 at 18:05
`a` is not a normal list it is a `pandas.Series` other than that your interpretation is correct — Joran Beasley, Jul 17 '13 at 18:06
Thanks everyone. To clarify, it's the first code snippet that's giving me the error... — Anne, Jul 19 '13 at 09:08

Marcin · Answer 1 · 2013-07-17T18:33:05.677

5

You've got the translation correct. However, the code you give cannot be run because a is a free variable.

My guess is that you are getting the error because you have assigned a list object to the name list. Don't do that, because list is a global name for the type of a list.

Also, in future please always provide a full stack trace, not just one part of it. Please also provide sufficient code that at least there are no free variables.

edited Jul 17 '13 at 18:33

answered Jul 17 '13 at 18:06

Marcin

48,559
18
128
201

@arshajii How do you figure? Try `[]()`. – Marcin Jul 17 '13 at 18:08
Apologies Marcin, I didn't want to repost the code from the original question, that's why I only cut out the snippet that was giving me a headache. You can see http://stackoverflow.com/a/16857854/2565842 for the whole code, it's just a couple of lines... – Anne Jul 19 '13 at 09:10

cwallenpoole · Answer 2 · 2013-07-17T18:19:14.230

If that is all of your code, then you have only a few possibilities:

myList.append is really a list
len is really a list
list is really a list
isnull is really a list
groupby is really a list
Series is really a list
The error exists somewhere behind groupby.

I'm going to go ahead and strike out myList.append (because that is impossible unless you are using your own groupby function for some reason) and Series. Unless you are importing Series from somewhere strange, or you are re-assigning the variable, we know Series can't be a list. A similar argument can be made for a.isnull.

So that leaves us with two real possibilities. Either you have re-assigned something somewhere in your script to be a list where it shouldn't be, or the error is behind groupby.

I think you're using the wrong groupby itertools.groupby takes and array or list as an argument, groupby in pandas may evaluate the first argument as a function. I especially think this because isnull() returns an array-like object.

Python conciseness confuses me

2 Answers2