2

Context: I was passing what I thought was a DataFrame, df.iloc[n], into a function. Thanks to the dialogue here I figured out this was causing errors because Pandas automatically converts a single row or column from a Dataframe into a series, and is easily solved by using df.iloc[[n]] instead of df.iloc[n].

Question: My question is why does Pandas do this? Is there some performance benefit in using Series instead of DataFrames? What is the reasoning behind this automatic conversion to a Series?

BLimitless
  • 2,060
  • 5
  • 17
  • 32
  • It's like asking why indexing 2d array converts it to a 1d array. It does not convert it to a Series, the row of the dataframe is logically a Series, not a dataframe. Technically speaking it's the difference between shapes of `(c,)` of `df.iloc[n]` and `(1, c)` of `df.iloc[[n]]` where `c` is a number of columns. – Yevhen Kuzmovych May 05 '21 at 16:15
  • I don't think what you're saying is quite right: `type(df.iloc[n])` is a Series, whereas `type(df.iloc[[n]])` is a DataFrame. This isn't just dimensionality: I can call `df.iloc[[n]].columns`, whereas `df.iloc[n].columns` throws an Error. There's real functionality change here, not just dimensionality change. My question is why does Pandas go through the effort to make this functionality change? There must be a reason for it. What is that reason? Thanks! – BLimitless May 05 '21 at 16:32

1 Answers1

2

As per Pandas documentation Why more than one data structure?

The best way to think about the pandas data structures is as flexible containers for lower dimensional data. For example, DataFrame is a container for Series, and Series is a container for scalars. We would like to be able to insert and remove objects from these containers in a dictionary-like fashion.

So, no conversion happening here, rather objects with different properties/methods being retrieved.

Laurent
  • 12,287
  • 7
  • 21
  • 37
  • 1
    This is perfect, thank you: "...objects with different properties/methods being retrieved" is exactly the insight I needed. – BLimitless May 05 '21 at 18:47