15

I've recently noticed that a function where I iterate over a DataFrame rows using .iloc is very slow. I found out that there's a faster method called .iat, that's said to be equivalent to .iloc. I tried it and it cut the run time down by about 75%.

But I'm a little hesitant: why is there an "equivalent" method that's faster? There must be some difference between the inner workings of these two and a reason why they both exist and not just the faster one. I've tried looking everywhere but even the pandas documentation just states that

DataFrame.iat
Fast integer location scalar accessor.

Similarly to iloc, iat provides integer based lookups. You can also set using these indexers.

And that doesn't help.

Are there limits to using .iat? Why is faster; is it sloppier? Or do I just switch to using .iat and happily forget .iloc ever existed?

Vladyslav
  • 2,018
  • 4
  • 18
  • 44
nirnroot
  • 674
  • 1
  • 6
  • 10
  • 3
    `.iat` working only with scalar, `.iloc` working with scalar + df, series -> more general -> slowier – jezrael Sep 26 '17 at 12:43
  • 1
    Cautionary note: if you are iterating over rows very often, the issue is not `at` vs ` loc` but rather that you should rarely be iterating over rows in the first place ;-) – JohnE Sep 26 '17 at 13:26

2 Answers2

20

iat and at working with scalar only, so very fast. Slower, more general functions are iloc and loc.

You can check docs:

Since indexing with [] must handle a lot of cases (single-label access, slicing, boolean indexing, etc.), it has a bit of overhead in order to figure out what you’re asking for. If you only want to access a scalar value, the fastest way is to use the at and iat methods, which are implemented on all of the data structures.

Similarly to loc, at provides label based scalar lookups, while, iat provides integer based lookups analogously to iloc.

Community
  • 1
  • 1
jezrael
  • 822,522
  • 95
  • 1,334
  • 1,252
7

iat and at gives only a single value output, while iloc and loc can give multiple row output.
Example:
iloc[1:2,5:8] is valid but iat[1:2,5:8] will throw error

Vismay
  • 101
  • 1
  • 5