Ignore column length when adding columns to a DataFrame

Question

I'm trying to add a bunch of returns to a DataFrame so that I can output it to a CSV file and do some basic graphing, but I'm getting an error message: ValueError: Length of values does not match length of index. I don't particularly care how long the columns are, but I was curious if it would be possible to assuage the index so that the first element in the list appears at index 0, and then everything else appears as subsequent values with NaN padding at the end if necessary. Is this doable in pandas?

for idx, col in enumerate(df.T.index.values):
    print "On " + str(idx) + " of " + str(len(df.T.index.values))
    startDate = dt.datetime.strptime(col.split()[0], "%m/%d/%y")
    endDate = startDate + dt.timedelta(days=91)

    returns1 = getReturnsForPeriod(df[col][1], startDate, endDate)
    retval[(col.split()[0], df[col][1], df[col][0])] = [ret for ret in returns1]

    returns2 = getReturnsForPeriod(df[col][2], startDate, endDate)
    retval[(col.split()[0], df[col][2], df[col][0])] = [ret for ret in returns2]

    returns3 = getReturnsForPeriod(df[col][3], startDate, endDate)
    retval[(col.split()[0], df[col][3], df[col][0])] = [ret for ret in returns3]

    returns4 = getReturnsForPeriod(df[col][4], startDate, endDate)
    retval[(col.split()[0], df[col][4], df[col][0])] = [ret for ret in returns4]

I think I need more explanation for what you're trying to do, but `df.reset_index()` is probably what you're looking for. — exp1orer, Nov 11 '14 at 19:54

score 1 · Accepted Answer · edited May 23 '17 at 12:09

1

I think this is a similar question to yours. Do it by going via dictionary first so you will know the final max length of your arrays.

Here just the basic mockup of the idea, not tested yet:

d = {}
for idx, col in enumerate(df.T.index.values):
  d.update(idx,col)
df = pandas.DataFrame.from_dict({idx,Series(col)) for idx,col in d.iteritems()})

edited May 23 '17 at 12:09

Community

1
1

answered Nov 12 '14 at 08:39

PlagTag

6,107
6
36
48

1

This was the (modified) solution that I found for the problem, but I forgot to accept your answer. Thank you. – weskpga Dec 30 '14 at 23:16

Ignore column length when adding columns to a DataFrame

1 Answers1