How to run a function to each row of Dataframe in Python

Question

I would like to loop through a function on each row of my dataset df. df is 920 x 10080. The function extracts the first 5 frequency components from a wave(which is formed from each row of data) using concept of Fast Fourier Transformation. The code for function:

def get_fft_values(y_values, T, N, f_s):
    f_values = np.linspace(0.0, 1.0/(2.0*T), N//2)
    fft_values_ = fft(y_values)
    fft_values = 2.0/N * np.abs(fft_values_[0:N//2])
    return f_values, fft_values  #f_values - frequency(length = 5040) ; fft_values - amplitude (length = 5040)   

t_n = 1
N = 10080
T = t_n / N
f_s = 1/T

f_values, fft_values = get_fft_values(df, T, N, f_s)

print(f_values[0:5], fft_values[0:5]

This is a function of Fourier transformation using one row of data as Dataframe(I sliced df = df.iloc[1,:]). I'm not sure how to run this function one each row. I tried

for row in df.iterrows():

  t_n = 1
  N = 10080
  T = t_n / N
  f_s = 1/T

  f_values, fft_values = get_fft_values(pd.DataFrame(row), T, N, f_s)

  plt.plot(f_values, fft_values, 'x', color='blue', markersize = '3')
  plt.xlabel('Frequency [Hz]', fontsize=16)
  plt.ylabel('Amplitude', fontsize=16)
  plt.title("Frequency domain of the data", fontsize=16)
  plt.show()

  print(f_values[0:5], fft_values[0:5], '\n')

which returned error

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-158-866f17cf6143> in <module>()
     51   f_s = 1/T
     52 
---> 53   f_values, fft_values = get_fft_values(pd.DataFrame(row), T, N, f_s)
     54 
     55   plt.plot(f_values, fft_values, 'x', color='blue', markersize = '3')

4 frames
/usr/local/lib/python3.6/dist-packages/numpy/core/_asarray.py in asarray(a, dtype, order)
     83 
     84     """
---> 85     return array(a, dtype, copy=False, order=order)
     86 
     87 

ValueError: setting an array element with a sequence.

and tried:

results = pd.DataFrame([get_fft_values(*x,T, N, f_s) for x in df.values.tolist()])
results = df.apply(lambda x: get_fft_values(*x))

which returned error

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-154-241b17a458df> in <module>()
----> 1 results = pd.DataFrame([get_fft_values(*x,T, N, f_s) for [x] in df.values.tolist()])
      2 results = df.apply(lambda x: get_fft_values(*x))
      3 results

<ipython-input-154-241b17a458df> in <listcomp>(.0)
----> 1 results = pd.DataFrame([get_fft_values(*x,T, N, f_s) for [x] in df.values.tolist()])
      2 results = df.apply(lambda x: get_fft_values(*x))
      3 results

ValueError: too many values to unpack (expected 1)

and tried

df = df.iloc[:,0:10080]

def get_fft_values(y_values, T, N, f_s):
    f_values = np.linspace(0.0, 1.0/(2.0*T), N//2)
    fft_values_ = fft(y_values)
    fft_values = 2.0/N * np.abs(fft_values_[0:N//2])
    return f_values, fft_values  #f_values - frequency(length = 5040) ; fft_values - amplitude (length = 5040)   

t_n = 1
N = 10080
T = t_n / N
f_s = 1/T

f_values, fft_values = get_fft_values(df, T, N, f_s)


df.apply(get_fft_values, axis=1)

which returned

--------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-178-75f82bbfdaca> in <module>()
     16 
     17 
---> 18 df.apply(get_fft_values, axis=1)

3 frames
/usr/local/lib/python3.6/dist-packages/pandas/core/apply.py in apply_series_generator(self)
    319             try:
    320                 for i, v in enumerate(series_gen):
--> 321                     results[i] = self.f(v)
    322                     keys.append(v.name)
    323             except Exception as e:

TypeError: ("get_fft_values() missing 3 required positional arguments: 'T', 'N', and 'f_s'", 'occurred at index 0')

This seems like a straightforward question but I am not sure how to fix it. How to write the correct for-loop in this case? Thanks a lot

@AMC I searched numpy and pandas docs, online blogs and other so posts — nilsinelabore, Jan 29 '20 at 05:53
Does this answer your question? [How to apply a function to two columns of Pandas dataframe](https://stackoverflow.com/questions/13331698/how-to-apply-a-function-to-two-columns-of-pandas-dataframe) — AMC, Jan 29 '20 at 05:56
Look at the docs for `.apply()`, or the link post. It’s just a matter of getting the right arguments from each row. — AMC, Jan 29 '20 at 07:06

score 3 · Accepted Answer · edited Jun 20 '20 at 09:12

3

For the question how to apply a function on each row in a dataframe, i would like to give a simple example so that you can change your code accordingly.

df = pd.DataFrame(data) ## creating a dataframe

def select_age(row): ##  a function which selects and returns only the names which have age greater than 18. Here the argument is row
                    ## i simply named it as row. So that its easy to understand that each row is being passed.
    boole = 'No'
    age  = row['Age']  ## from each row we are extracting the values of age 
    if age >18:
        boole = 'Yes'
    return boole  ## if age is greater, then we are passing yes for that row

df['Age>18'] = df.apply(lambda x: select_age(x), axis =1) # here we are applying apply function which performs the required action on each row

"""For each row we are passing yes or no and storing it in a new column"""

output

edited Jun 20 '20 at 09:12

Community

1
1

answered Jan 29 '20 at 04:50

Srikar Manthatti

504
2
15

I think the lambda is unnecessary here. – AMC Jan 29 '20 at 15:44
Thanks for your answer. My currently problem is how to apply the function when there are multiple arguments? – nilsinelabore Jan 30 '20 at 00:42
try looking at `functools.partial()` – Brad Jul 23 '21 at 20:20

How to run a function to each row of Dataframe in Python

1 Answers1