Pandas columns line intersection

Question

I've got a pandas dataframe where each column represents the y values of a descending line from 1 to 0 and the indexes represent the x values. Now I'm interested in finding the intersection points between these lines and a given constant (linearly interpolated).

Example of the desired output:

>>> df = pd.DataFrame({'y1': [1, .7, .4, .1, 0],
                       'y2': [1, .9, .7, .3, 0],
                       'y3': [1, .6, .3, .2, 0],
                       'y4': [1, .7, .5, .3, 0]}, index=[0, 10, 20, 30, 40])
>>> xs = get_intersection(df, .5)
>>> xs
{'x1': 16.6667,  # = scipy.interpolate.interp1d([.7, .4], [10, 20])(.5)
 'x2': 25.0,     # =                   interp1d([.7, .3], [20, 30])(.5)
 'x3': 13.3332,  # =                   interp1d([.6, .3], [10, 20])(.5)
 'x4': 20}       # No interpolation required

My data consists of roughly 400 rows and 50.000 columns.

possible solution:

I found this SO answer that finds the intersection points between two lines with the following method:

idx = np.argwhere(np.diff(np.sign(f - g)) != 0).reshape(-1) + 0

I think this can be adjusted to work with my dataframes, but I'm not sure how to proceed from here:

>>> idx = np.argwhere(np.diff(np.sign(df - .5), axis=0) != 0)
>>> idx
array([[1, 0],
       [1, 2],
       [1, 3],
       [2, 1],
       [2, 3]], dtype=int64)

Since people seem to misunderstand the question, I'm interested in finding these points:

Which can be found by linearly interpolating the two nearest points.

Solution: B. M. gave me a step in the right direction:

def get_intersection(df, c):
    dfind = len(df) - df.loc[::-1].apply(np.searchsorted, args=(c,), raw=True)

    result = {}
    for i, v in enumerate(dfind):
        result[df.columns.values[i]] = interp1d([df.iloc[v][i], df.iloc[v - 1][i]], [df.index[v], df.index[v - 1]])(.5)

    return result


>>> get_intersection(df, .5)
{'y1': array(16.666666666666668), 'y2': array(25.0), 'y3': array(13.333333333333332), 'y4': array(20.0)}

I'm not sure how I can use `idx` to find the values of `ys`. Or if the method to get `idx` is needlessly complicated, another way to calculate `ys`. — Jurgy, Dec 05 '17 at 14:35
You can find the indices of the first point by `dfind=len(df)-df.loc[::-1].apply(np.searchsorted,args=(0.5,),raw=True)` . Atter that do it by a loop . — B. M., Dec 05 '17 at 14:58

B. M. · Accepted Answer · 2017-12-05T15:59:14.033

since you need 1D function (interp1d), loops are as clear, easy to read, and speed as pandas functions. use np.searchsorted to find the good segment :

d = dict()
ind=df.index.values
for k,v in df.iteritems():
    y=v.values
    i=len(y)-np.searchsorted(y[::-1],0.5) #indices
    sl = slice(i-1,i+1)
    d[k]= +scipy.interpolate.interp1d(v[sl],ind[sl])(.5)

print (pd.Series(d))

# y1    16.666667
# y2    25.000000
# y3    13.333333
# y4    20.000000
# dtype: float64

Probably this must be tuned if no intersection point exists.

Pandas columns line intersection

1 Answers1