How do I correctly use .loc in pandas?

Question

I am a newbie and trying to figure out how to correctly use the .loc function in pandas for slicing a dataframe. Any help is greatly appreciated.

The code is:

df1['Category'] = df[key_column].apply(lambda x: process_df1(x, 'category'))

where df1 is a dataframe, key_column is a specific column identified to be operated upon process_df1 is a function defined to run on df1.

The problem is I am trying to avoid the error: "A value is trying to be set on a copy of a slice from a DataFrame. Try using

.loc[row_indexer,col_indexer] = value instead"

I don't want to ignore / suppress the warnings or set `pd.options.mode.chained_assignment = None.

Is there an alternative besides these 2?

I have tried using

df.loc[df1['Category'] = df[key_column].apply(lambda x: process_df1(x, 'category'))]

but it still produces the same error. Am I using the .loc incorrectly?

Apologies if it is a confusing question.

df1 = df[:break_index] df2 = df[break_index:]

Thank you.

Hi you could try list comprehension like this df1['Category'] = [process_df1(x,'category') for x in range(len(df1)] — Tom, Jan 08 '21 at 02:52
share your sample data, share expected results and there may be a better way to address the problem. — Joe Ferndz, Jan 08 '21 at 02:58
Hey Tom, thank you for your reply. I corrected the syntax and now the function doesn't run at all. Sorry. — Anurag Srivastava, Jan 08 '21 at 02:59
can you do df.head(10) and post that so we know what kind of data you are dealing with and how we need to use process_df1 — Joe Ferndz, Jan 08 '21 at 03:10
what is `x1` in `process_df1` ? I cant find any reference of it — Joe Ferndz, Jan 08 '21 at 03:19
@Jeril could you verify if .loc method is being used correctly? I have many results on google that suggest using .loc will get rid of this warning. — Anurag Srivastava, Jan 08 '21 at 03:30
The problem is not `apply`, on its own. Rather that `df1` is a part of a bigger dataframe and you are trying to modify just it. Your code doesn't show where and how you define `df1`. But you can try: `df.loc[df1.index, 'Category'] = df[key_column].apply(lambda x: process_df1(x, 'category'))]` where `df` is the big dataframe from which you slice `df1`. — Quang Hoang, Jan 08 '21 at 03:40
df is broken down into df1 and df2 based on an index. so df1 = df[:break_index] and df2=[break_index:] — Anurag Srivastava, Jan 08 '21 at 04:07

score 0 · Answer 1 · answered Jan 08 '21 at 02:58

0

The apply method performs the function in place to the series you are running it on (key_column in this case)

If you are trying to create a new column based upon a function using another column as input you can use list comprehension

df1['Category'] = [process_df1(x, 'category') for x in df1[key_column]]

NOTE I'm assuming process_df1 operates on a single value from the key_column column and returns a new value based upon your writing. If that's not the case please update your question

answered Jan 08 '21 at 02:58

sedavidw

11,116
13
61
95

I have updated the question per your request. Could you please take a look now? – Anurag Srivastava Jan 08 '21 at 03:08
Also, with your code Sir, I still get the warning - "A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead" – Anurag Srivastava Jan 08 '21 at 03:14

score 0 · Answer 2 · answered Jan 08 '21 at 03:37

Unless you give more details on the source data and your expected results, we won't be able to provide you clear answer. For now, here's something I just created to help you understand how we can pass two values and get things going.

import pandas as pd

df = pd.DataFrame({'Category':['fruit','animal','plant','fruit','animal','plant'],
                   'Good' :[27, 82, 32, 91, 99, 67],
                   'Faulty' :[10, 5, 12, 8, 2, 12],
                   'Region' :['north','north','south','south','north','south']})

def shipment(categ,y):
    d = {'a': 0, 'b': 1, 'c': 2, 'd':3}

    if (categ,y) == ('fruit','a'):
        return 10
    elif (categ,y) == ('fruit','b'):
        return 20
    elif (categ,y) == ('animal','a'):
        return 30
    elif (categ,y) == ('animal','c'):
        return 40
    elif (categ,y) == ('plant','a'):
        return 50
    elif (categ,y) == ('plant','d'):
        return 60
    else:
        return 99

df['result'] = df['Category'].apply(lambda x: shipment(x,'a'))
print (df)

How do I correctly use .loc in pandas?

2 Answers2