Minimize function with Pandas dataframe

Question

I have to find the 2 input values for which the output value is minimized:

import pandas as pd

def calc_func(x1, x2):
    var1 =  pd.DataFrame([x1]).loc[0]
    var2 =  pd.DataFrame([x2]).loc[0] 
    y = var1-var2
    return(y)

from scipy.optimize import minimize
x0 = [1,2,3]
res = minimize(calc_func,x0,x0, method='BFGS')

however this gives me the following error

ValueError: setting an array element with a sequence.

Which can be explained by the fact that the calculations with pandas dataframes use single numbers instead of arrays...

What is the best way to minimize my function?

Comments:

Unfortunately it is not an option to completeley remove all pandas calculations from the function.

So basically the error is in the hidden code? How can one *minimize* it if one does not know how it looks like? — Quang Hoang, Apr 30 '19 at 11:46
I changed the hidden code with some fake code. However, in reality the code is more complex, and makes sense to minimize. I hope the fake code will clarify my question. — Koot6133, Apr 30 '19 at 11:56
You might be interested in [this](https://stackoverflow.com/a/55891243/1534017) and [this](https://stackoverflow.com/a/43241280/1534017) answer. — Cleb, Apr 30 '19 at 12:14
According to the documentation https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html fucntion `calc_func` should return a single float value. Right now `calc_func` returns pandas series which is a collection of many values. — , Apr 30 '19 at 12:14
@Poolka: Not exactly right, it is true that the object is pandas series, but it's not many values. He needs to return a float though, which is easily fixed but not the source of the problem — Michael Heidelberg, Apr 30 '19 at 12:22

Michael Heidelberg · Accepted Answer · 2019-04-30T12:43:35.850

The function minimize minimizes a function from R^n to R. The simplest thing to do, is to have x,y both concatenated in a single vector z, then optimize the function with respect to z. The following code works:

import pandas as pd
from scipy.optimize import minimize
import numpy as np

def calc_func(x):
    return(x[0]-x[1])

x1 = [1,2,3]
x2 = [3,4,5]
v1 = pd.DataFrame([x1]).values[0]
v2 = pd.DataFrame([x2]).values[0]
x = np.array([v1,v2])
res = minimize(calc_func,x, method='BFGS')

If you really need to optimize a function from R^n to R^m then you need to use another method (which I did not find by quickly looking in the docs).

Minimize function with Pandas dataframe

1 Answers1