pandas iterrows index seems wrong(misunderstanding!)

Asked May 18 '20 at 15:22

Active May 18 '20 at 15:22

Viewed 151 times

I am new to pandas and I found this weird problem I could not get my head around (hate pandas). I have a dataframe df with shape: (1033, 5). So, my understanding is that there are 1033 rows and 5 columns.

Now I iterate through this using:

for idx, row in df.iterrows():
    print(idx)

In this loop I see numbers greater than 1033 such as 1722.. WTF is going on? Any pointers would be great.

asked May 18 '20 at 15:22

JohnJ

6,736
13
49
82

1

Probably the index is not a normal range, and some values are missing – yatu May 18 '20 at 15:23
1

Check with `(pd.RangeIndex(df.shape[0])==df.index).all()` – yatu May 18 '20 at 15:24
I get `AttributeError: 'NoneType' object has no attribute 'all'` for that – JohnJ May 18 '20 at 15:26
Well then, df.index.max() – yatu May 18 '20 at 15:27
yes. it is giving me 1722 :( WTF is this? :( How do I refer to this row in pandas now? :( – JohnJ May 18 '20 at 15:31
No worries :) reset the index, `df=df.reset_index(drop=True)` – yatu May 18 '20 at 15:32
can you please tell me when I need to aware of this? why is the index this way? this seems like a nasty gotcha – JohnJ May 18 '20 at 15:33
Nah this is probably that you've dropped some rows, reindexed at some point etc. Keeping the original index can be useful in some cases – yatu May 18 '20 at 15:34
ah ok. Thanks. one more Q: what is `drop=True` do? Please post this as an answer and I will accept it! – JohnJ May 18 '20 at 15:35
Nah this is a clear dupe. That just drops the original index, otherwise it adds it as column – yatu May 18 '20 at 15:36
ah Ok - I could find a post for it. Anyways. Thanks a tonne. – JohnJ May 18 '20 at 15:37
No problem, will set as a reset_index dupe – yatu May 18 '20 at 15:38
This index is not the regular index as you would imagine in a loop, it is the index position of that data in the dataframe – Pianistprogrammer Feb 21 '22 at 16:07

pandas iterrows index seems wrong(misunderstanding!)

0 Answers0