How to speed up list appending from dataframe.loc operation

Question

I am trying to get the index of first occurance of each unique component names and then appending the indices to the list.

I have dataframe with around 20k rows.

 mylist=[]

 for i in df['name']:
        mylist.append(df.loc[df.name==i].index[0])
 mylist=set(mylist)

How can I speed up the above process? It takes around a minute to append to the list from dataframe.

Please improve your question, so that we can reproduce your issue. How to create [mcve] Thanks @YasharAhmadov: Screenshot is only second best solution, text/code would be much better — HedgeHog, Nov 18 '21 at 20:39
Please explain what you're trying to achieve here. It looks a lot like if you're just extracting first level of index grouped by name - which should be a one-liner in pandas — Marat, Nov 18 '21 at 20:40
https://stackoverflow.com/questions/20109391/how-to-make-good-reproducible-pandas-examples — SuperStormer, Nov 18 '21 at 20:41
I am trying to get the index of first occurance for each unique name in dataframe — Elgun Valiyev, Nov 18 '21 at 20:45

Marat · Accepted Answer · 2021-11-18T21:14:24.460

1

Can't test without a data example, but here it is:

df.reset_index().groupby('name').first()['index'].to_list()

A minimal reproducible example would look like this:

pd.DataFrame({'name': ['ABBA', 'LZ', 'LZ', 'LZ', 'IronMaiden', 'PinkFloyd', 'LZ', 'PinkFloyd']})

DataFrame:

         name
0        ABBA
1          LZ
2          LZ
3          LZ
4  IronMaiden
5   PinkFloyd
6          LZ
7   PinkFloyd

Desired outcome:

[0, 1, 4, 5]

edited Nov 18 '21 at 21:14

answered Nov 18 '21 at 20:51

Marat

added a picture – Elgun Valiyev Nov 18 '21 at 20:55
@ElgunValiyev please add text or code instead – Marat Nov 18 '21 at 20:56
your code gives the list for the names of unique components rather than their index. – Elgun Valiyev Nov 18 '21 at 20:58
@ElgunValiyev right. This is why a test example is needed – Marat Nov 18 '21 at 20:59
this loads from csv file thats why i cant provide here – Elgun Valiyev Nov 18 '21 at 21:02
@ElgunValiyev that'd not be helpful at all. A good practice is to post a minimal reproducible example, a small made-up dataframe and the desired result – Marat Nov 18 '21 at 21:04
you are right. basicly I want to get the indices of first occurances for each name from original dataframe. even if as_index=False is set inside groupby it gives the indices from newly created groupby dataframe as 1 2 3 4 5 6 7 etc. – Elgun Valiyev Nov 18 '21 at 21:07
@ElgunValiyev updated the answer, added what a minimal reproducible example should look like. Note - it's your responsibility to add something like this to the question – Marat Nov 18 '21 at 21:09
thanks it works as desired. – Elgun Valiyev Nov 18 '21 at 21:19

1 Answers1