Use variable in for loop as dataframe column name

Question

I am attempting to use a for loop iterate through a list of column names and on each iteration create a new column based on a string prefix and the original column name (stored in the list). the problem is that the dataframe thinks the variable is a series in the frame when

cols_to_transform = ['Price','Km_From_CBD','Local_Median_Price', 'AreaSize']

for x in cols_to_transform:
    df.x #This is where the problem
    df[x] = df[x>1]
    newcolname = ('T1_Pre_'+ x)
    df.newcolname = df.x + 1 #and same problem here

(DataFrame' object has no attribute 'x')

(DataFrame' object has no attribute 'newcolname' )

Ideally I would have a "global variable" or parameter that overrides the expected pandas object and takes the contents of the variable as a column name rather than the variable itself.

I cannot use pandas apply as in addition to creating the columns I need to create a series of subplots showing the change in transformation.

I know I can cast the whole line of code to string and then use exec, but I'm getting really over doing this as it feels like a cheap work around.

Thanks in Advance! Also it's my first question, go easy on me :)

score 5 · Answer 1 · answered Jun 05 '18 at 08:43

5

Try this code:

cols_to_transform = ['Price','Km_From_CBD','Local_Median_Price', 'AreaSize']

for x in cols_to_transform:
    df[x]

    newcolname = ('T1_Pre_'+ x)
    df.newcolname = df[x] + 1

answered Jun 05 '18 at 08:43

Ernest S Kirubakaran

1,524
12
16

Yes this is a good solution to the problem I posted however it creates other problems inside my actual code. Inside the loop I have: df[x] = df[[x] >1] which does not work. I know there are two different methods of accessing pandas data frames ( df.name and df['name'] ) I believe this uses the second method. Do you know a way I can perform this type of operation using your method? (I will update the question). Thank you – Dominic Jun 05 '18 at 10:14
Can you please elaborate what are you trying to achieve using the command: df[x] = df[[x] >1] ? – Ernest S Kirubakaran Jun 05 '18 at 10:37
I'm doing a boxcox transformation on the columns and creating a new column with the result of that transformation there are some columns that have value 0, which box cox doesn't like. Ultimately I was asking about what jpp marked as duplicate – Dominic Jun 05 '18 at 10:47

Use variable in for loop as dataframe column name

1 Answers1