I have a table with 100 columns and 1M records and loaded into Panda DF.
Example of records:
C1 C2 C3 ...
A 1 2
A 1 2
B 2 2
B 2 2
C 3 2
I want to make unique records based on 2 columns (C1 and C2) - basically I know that if 2 of them are the same - the rest of the columns are the same.
Result would be:
C1 C2 C3 ...
A 1 2
B 2 2
C 3 2
How to achieve this?
Thanks.