Apply elementwise, concatenate resulting rows into a DataFrame

Question

I have a list of values (could easily become a Series or DataFrame), on which I want to apply a function element-wise.

x = [1, 5, 14, 27]

The function itself returns a single row of a DataFrame (returning the original value x and two result value columns), and I want to end up with a single DataFrame.

    x    Val1    Val2
0   1       4      23
1   5      56      27
2  14      10       9
3  27       8      33

The simple method is a for loop over the list and row bind the results with df.append(), but I'm sure there is a way to do this with the .apply()family of functions. I just can't figure out exactly which to use. I'm pretty familiar with doing this type of thing in R, and am familiar with Python, just need to get my head around the pandas syntax.

EDIT: More concrete example for clarity

Example function:

def returnsquares(x):
    return pd.DataFrame({"input": [x], "sq": x**2, "cube": x**3})

Input of the function is a scalar, output is a DataFrame with a single row (Not a series).

Code that works:

result = pd.DataFrame({}, columns=["input", "sq", "cube"])
for entry in x:
    result = result.append(returnsquares(entry))

(The values of the output are obviously not the same as above, but are the same shape). Is there a better method for doing this?

piRSquared · Accepted Answer · 2016-12-16T21:24:35.503

2

consider the following function that returns the same stuff you show in your example

def special_function(x):
    idx = ['x', 'Val1', 'Val2']
    d = {
         1: pd.Series([x, 4, 23], idx),
         5: pd.Series([x, 56, 27], idx),
        14: pd.Series([x, 10, 9], idx),
        27: pd.Series([x, 8, 33], idx),
    }
    return d[x]

then you want to combine into a single dataframe using pd.DataFrame.from_records

pd.DataFrame.from_records([special_function(i).squeeze() for i in x])

Or use pd.concat

pd.concat([special_function(i) for i in x])

Or make x a series and use apply

x = pd.Series([1, 5, 14, 27])
x.apply(lambda y: special_function(y).iloc[0], 1)

Be aware of timings
timing
Don't fear the list comprehension

edited Dec 16 '16 at 21:24

answered Dec 16 '16 at 19:56

piRSquared

285,575
57
475
624

But my function returns a DataFrame with a single row, not a series (I didn't write the function, so take it as a given). When I run this approach with my code, I get a DataFrame that is the right size and shape, but with values of what should be the column names. I'll edit the question to clarify. – Connor J Dec 16 '16 at 20:18
So, this works! Thanks! But I just have a gut feeling that a more elegant solution exists. It's definitely a step up in terms of not having to initialize the vector, but a list comprehension still feels like using a loop rather than exploiting the optimized structure of DataFrames and Series. But I could be wrong, and this is just how it goes in python. – Connor J Dec 16 '16 at 20:42
I am more hoping to eliminate the list comprehension than anything else, it just seems counter to this type of data structure. I'm thinking something *like* `x.apply(special_function, axis=1)`. This particular thing doesn't work, but that's more the type of solution I had in mind. This is good though, thanks! – Connor J Dec 16 '16 at 21:21
(Where x is a Series or DataFrame, not a list) – Connor J Dec 16 '16 at 21:21

Apply elementwise, concatenate resulting rows into a DataFrame

1 Answers1

Linked