2

I want to apply a function using one column on another column in a pandas dataframe

# function to select an item from a list in a column
def select_index(r, i):
    return list(np.take(r, i))

# create DataFrame
col = ['index', 'Column2', 'Column3']

d = {'index': [['a1', 'a2', 'a3'], ['a10', 'a20', 'a30']],
     'Column2': [['b1', 'b2', 'b3'], ['b10', 'b20', 'b30']],
     'Column3': [[0, 1], [1, 2]]
    }

df = pd.DataFrame(data=d, columns=col)
df.set_index('index', inplace=True)

print(df)

                         Column2 Column3
index                                   
[a1, a2, a3]        [b1, b2, b3]  [0, 1]
[a10, a20, a30]  [b10, b20, b30]  [1, 2]

When I do the apply like:

df['Column2'] = df[['Column2', 'Column3']].apply(lambda x: select_index(*x), axis=1)

I get the following error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-241-60f4b4130a38> in <module>
      1 df.loc = df[['Column2', 'Column3']].apply(
      2     lambda x: select_index(*x),
----> 3     axis=1
      4 )

~/miniconda3/envs/cl/lib/python3.6/site-packages/pandas/core/frame.py in apply(self, func, axis, broadcast, raw, reduce, result_type, args, **kwds)
   6012                          args=args,
   6013                          kwds=kwds)
-> 6014         return op.get_result()
   6015 
   6016     def applymap(self, func):

~/miniconda3/envs/cl/lib/python3.6/site-packages/pandas/core/apply.py in get_result(self)
    140             return self.apply_raw()
    141 
--> 142         return self.apply_standard()
    143 
    144     def apply_empty_result(self):

~/miniconda3/envs/cl/lib/python3.6/site-packages/pandas/core/apply.py in apply_standard(self)
    246 
    247         # compute the result using the series generator
--> 248         self.apply_series_generator()
    249 
    250         # wrap results

~/miniconda3/envs/cl/lib/python3.6/site-packages/pandas/core/apply.py in apply_series_generator(self)
    274         else:
    275             try:
--> 276                 for i, v in enumerate(series_gen):
    277                     results[i] = self.f(v)
    278                     keys.append(v.name)

~/miniconda3/envs/cl/lib/python3.6/site-packages/pandas/core/apply.py in <genexpr>(.0)
    365         constructor = self.obj._constructor_sliced
    366         return (constructor(arr, index=self.columns, name=name)
--> 367                 for i, (arr, name) in enumerate(zip(self.values,
    368                                                     self.index)))
    369 

~/miniconda3/envs/cl/lib/python3.6/site-packages/pandas/core/series.py in __init__(self, data, index, dtype, name, copy, fastpath)
    279         generic.NDFrame.__init__(self, data, fastpath=True)
    280 
--> 281         self.name = name
    282         self._set_axis(0, index, fastpath=True)
    283 

~/miniconda3/envs/cl/lib/python3.6/site-packages/pandas/core/generic.py in __setattr__(self, name, value)
   4396             object.__setattr__(self, name, value)
   4397         elif name in self._metadata:
-> 4398             object.__setattr__(self, name, value)
   4399         else:
   4400             try:

~/miniconda3/envs/cl/lib/python3.6/site-packages/pandas/core/series.py in name(self, value)
    406     def name(self, value):
    407         if value is not None and not is_hashable(value):
--> 408             raise TypeError('Series.name must be a hashable type')
    409         object.__setattr__(self, '_name', value)
    410 

TypeError: Series.name must be a hashable type

I don't really understand why this is happening.

Any possible solutions to fix this?

Edit: I want the resulting dataframe to be like:

                         Column2 Column3
index                                   
[a1, a2, a3]        [b1, b2]  [0, 1]
[a10, a20, a30]  [b20, b30]  [1, 2]
cs95
  • 379,657
  • 97
  • 704
  • 746
armundle
  • 1,149
  • 2
  • 15
  • 28

2 Answers2

4

This seems to be an issue with your index being mutable objects. Things begin to work if you reset it first.

df.reset_index().apply(lambda x: select_index(x['Column2'], x['Column3']), axis=1)

0      [b1, b2]
1    [b20, b30]
dtype: object

Or,

df.reset_index()[['Column2', 'Column3']].apply(lambda x: select_index(*x), axis=1)

0      [b1, b2]
1    [b20, b30]
dtype: object

df['Column4'] = df.reset_index()[['Column2', 'Column3']].apply(
       lambda x: select_index(*x), axis=1).values
df

                         Column2 Column3     Column4
index                                               
[a1, a2, a3]        [b1, b2, b3]  [0, 1]    [b1, b2]
[a10, a20, a30]  [b10, b20, b30]  [1, 2]  [b20, b30]
cs95
  • 379,657
  • 97
  • 704
  • 746
2

why not just using for loop here

[select_index (x, y )for x,y in zip(df['Column2'], df['Column3'])]
Out[314]: [['b1', 'b2'], ['b20', 'b30']]
BENY
  • 317,841
  • 20
  • 164
  • 234
  • 1
    Why not? Because this is pandas and python. – JE_Muc Mar 15 '19 at 15:10
  • 3
    @Scotty1- [There are situations where loops are preferable over other pandas functions.](https://stackoverflow.com/questions/54028199/for-loops-with-pandas-when-should-i-care) I feel this is a good use case. – cs95 Mar 15 '19 at 15:11
  • 3
    @Scotty1- See also [this discussion](https://stackoverflow.com/questions/54432583/when-should-i-ever-want-to-use-pandas-apply-in-my-code) on the applicability of `apply`. – cs95 Mar 15 '19 at 15:11
  • @coldspeed Thanks for the links. Indeed looks like a good approach for the given dataset. – JE_Muc Mar 15 '19 at 15:14
  • @Scotty1- so , you think we should not using for loop here ? – BENY Mar 15 '19 at 15:20
  • 2
    Nope, my last comment intended to say that I agree that for loops may be a good approach in this case. :) – JE_Muc Mar 15 '19 at 15:34