Sort pandas df column by a custom list of values

Question

With this df, I want to sort a column based on a custom list:

pd.DataFrame(
{'id':[2967, 5335, 13950, 6141, 6169],\
 'Player': ['Cedric Hunter', 'Maurice Baker' ,\
            'Ratko Varda' ,'Ryan Bowen' ,'Adrian Caldwell'],\
 'Year': [1991 ,2004 ,2001 ,2009 ,1997],\
 'Age': [27 ,25 ,22 ,34 ,31],\
 'Tm':['CHH' ,'VAN' ,'TOT' ,'OKC' ,'value_not_present_in_sorter'],\
 'G':[6 ,7 ,60 ,52 ,81]})

    Age G   Player  Tm  Year    id
0   27  6   Cedric Hunter   CHH 1991    2967
1   25  7   Maurice Baker   VAN 2004    5335
2   22  60  Ratko Varda TOT 2001    13950
3   34  52  Ryan Bowen  OKC 2009    6141
4   31  81  Adrian Caldwell value_not_present_in_sorter 1997    6169

say this is the custom list:

sorter = ['TOT', 'ATL', 'BOS', 'BRK', 'CHA', 'CHH', 'CHI', 'CLE', 'DAL','DEN',\
          'DET', 'GSW', 'HOU', 'IND', 'LAC', 'LAL', 'MEM', 'MIA', 'MIL',\
          'MIN', 'NJN', 'NOH', 'NOK', 'NOP', 'NYK', 'OKC', 'ORL', 'PHI',\
          'PHO', 'POR', 'SAC', 'SAS', 'SEA', 'TOR', 'UTA', 'VAN',\
          'WAS', 'WSB']

I know about the answer here: sorting by a custom list in pandas

which provides this solution:

df.Tm = df.Tm.astype("category")
df.Tm.cat.set_categories(sorter, inplace=True)

df.sort_values(["Tm"])

but that answer is 4 years old and the values in the column that are not present in sorter are replaced by nans (which I don't want). I know I could probably use .unique() and append to the end of the list.

So my question: is there a better way to do a custom sort today by using pandas new in built capabilities and if to keep all values from a column and not replace them by nans, is there a better solution than:

other_values = set(df["TM"].unique()) - set(sorter)
sorter.append(other_values)

I think the solution you provide sounds reasonable and is probably the best you will do. Although it seems you may want to swap your sets on that first line: `set(df["TM"].unique()) - set(sorter)`. But I'm not ruling out another possibility. Someone else might come up with one :) — busybear, Jan 15 '19 at 15:52

score 4 · Accepted Answer · answered Jan 15 '19 at 15:59

The solution you mention is a good starting point. You can use ordered=True with set_categories to ensure you set categorical ordering as required:

df['Tm'] = df['Tm'].astype('category')
not_in_list = df['Tm'].cat.categories.difference(sorter)
df['Tm'] = df['Tm'].cat.set_categories(np.hstack((sorter, not_in_list)), ordered=True)

df = df.sort_values('Tm')

print(df)

   Age   G           Player                           Tm  Year     id
2   22  60      Ratko Varda                          TOT  2001  13950
0   27   6    Cedric Hunter                          CHH  1991   2967
3   34  52       Ryan Bowen                          OKC  2009   6141
1   25   7    Maurice Baker                          VAN  2004   5335
4   31  81  Adrian Caldwell  value_not_present_in_sorter  1997   6169

Sort pandas df column by a custom list of values

1 Answers1

Linked