Different sized lists to dataframe

Question

I have the user input a list of minutes and then a specific size. I then have a function that calculates the average price difference for every time that size shows up and this value is added to a list. Thus, I will have as many lists as the length of minutes. For example, if the user inputs 5,10; then I have 2 lists. I then try to add the list into a data frame, but am unable to as the lists are different in size and get an error that : ValueError: Length of values does not match length of index

This is the code I have for trying to insert the list into the dataframe (i is which element in the minutes list; i.e for 5,10 I have i=0 for 5 and i=1 for 10 done in a loop) list1 is a list, list2 is a dataframe:

 list2.insert(i,i, list1)  
 export_csv = list2.to_csv(file2 ,index = None, header=False)
 list1=[]

The error comes for this line of code: list2.insert(i,i, list1)

Here is an example: List 1=[0.5,3.5,7.5] then I want to insert it into the data frame: list2.insert(i,i, list1) Then I want to empty the list. Once it goes through the function the next list will be List1 is now= [7, 0.5, 8, 51.5, 2] and I want to insert that into column 2, why I wrote list2.insert(i,i, list1)

Here is my full code if necessary:

#df is a data frame
#b is also a dataframe
#list2 is a dataframe
#list1 is a list

for i in range (0, len(df)):
            size=df.iloc[i,0]
            for i in range(0,len(numbers)-1):            
                for number in numbers:
                #for i in range (0, len(numbers)-1):
                    print(number)
                    for filename in filenames:
                        b['diff']=abs(b['price']-b['orig_price'])
                        list1.extend((b['diff']))
                        print('size', size, list1)                    
                    list2[i+size+number]=list1 
                    export_csv = list2.to_csv(file2 ,index = None, header=True)
                    list1=[]

Welcome to Stack Overflow! In order to get a good answer much more quickly, it would help if you added some sample data for your lists and your desired output. See: https://stackoverflow.com/help/minimal-reproducible-example — Max Power, Jan 30 '20 at 23:31
Does this answer your question? [add columns different length pandas](https://stackoverflow.com/questions/27126511/add-columns-different-length-pandas) — Stuart, Jan 30 '20 at 23:33
Dataframes are designed to have columns / rows all the same length. If you need different lengths it will pad them with `NaN` values. It might be useful to show what output you need to achieve (a CSV file?) — Stuart, Jan 30 '20 at 23:39
The document doesn't answer my question because I am unable to use `.concat()` because I do not know how many list's the user will input because it is running through a loop. — Monica, Jan 31 '20 at 18:36

dkhara · Accepted Answer · 2020-01-31T19:17:54.397

IIUC, you want to set some list as a new dataframe column.

The ValueError: Length of values does not match length of index is most likely coming up because you're trying to insert a list of different length to a dataframe. In other words, make sure the length of your list equals the the number of rows in your dataframe. Otherwise, you will keep receiving this error. If you want to see a slightly more efficient way of creating new columns, keep reading.

Let's start with a sample list:

print(l)                                                                                                                        
[1, 2, 3]

And a sample dataframe:

print(df)                                                                                                                       
  c1  c2  c3
0  a   8   6
1  b   8   6
2  c   8   6

Then you can simply assign the list to a new column by:

df['new_lst_variable'] = l

print(df)                                                                                                                       
  c1  c2  c3  new_lst_variable
0  a   8   6                 1
1  b   8   6                 2
2  c   8   6                 3

Update

If you have a list that doesn't quite match up with the number of rows in your dataframe:

l2 = [1, 2, 3, 4]

You could use pandas.concat

df = pd.concat([df,pd.Series(l2)], ignore_index=True, axis=1)

print(df)                                                              
     0    1    2  3
0    a  8.0  6.0  1
1    b  8.0  6.0  2
2    c  8.0  6.0  3
3  NaN  NaN  NaN  4

You could also use DataFrame.fillna to fill these nans with whatever you'd like:

df = df.fillna(0)

print(df)                                                              
   0    1    2  3
0  a  8.0  6.0  1
1  b  8.0  6.0  2
2  c  8.0  6.0  3
3  0  0.0  0.0  4

The issue is the data frame will have different lengths because the data will vary based on what entries the user inputs and what values are found. Thus, the lists will be all different sizes. — Monica, Jan 31 '20 at 18:45
I've updated the answer. Does that look more like what you want? — dkhara, Jan 31 '20 at 19:13
Look at the line containing `df = df.fillna(0)`. This drops na values and replaces them with 0. You can replace 0 with any other value, and it will replace the nans with that value. — dkhara, Jan 31 '20 at 19:54

Different sized lists to dataframe

1 Answers1

Update