1

Say I have the following table:

+----+---------+--------+---------+---------+---------+---------+-----------+-----------+-----------+----------+-----------+------------+------------+---------+---+
|  1 | 0.72694 | 1.4742 | 0.32396 | 0.98535 |       1 | 0.83592 | 0.0046566 | 0.0039465 |   0.04779 |  0.12795 |  0.016108 |  0.0052323 | 0.00027477 |  1.1756 | 1 |
|  2 | 0.74173 | 1.5257 | 0.36116 | 0.98152 | 0.99825 | 0.79867 | 0.0052423 | 0.0050016 |   0.02416 | 0.090476 | 0.0081195 |   0.002708 | 7.48E-05   | 0.69659 | 1 |
|  3 | 0.76722 | 1.5725 | 0.38998 | 0.97755 |       1 | 0.80812 | 0.0074573 |  0.010121 |  0.011897 | 0.057445 | 0.0032891 | 0.00092068 | 3.79E-05   | 0.44348 | 1 |
|  4 | 0.73797 | 1.4597 | 0.35376 | 0.97566 |       1 | 0.81697 | 0.0068768 | 0.0086068 |   0.01595 | 0.065491 | 0.0042707 |  0.0011544 | 6.63E-05   | 0.58785 | 1 |
|  5 | 0.82301 | 1.7707 | 0.44462 | 0.97698 |       1 | 0.75493 |  0.007428 |  0.010042 | 0.0079379 | 0.045339 | 0.0020514 | 0.00055986 | 2.35E-05   | 0.34214 | 1 |
|  7 | 0.82063 | 1.7529 | 0.44458 | 0.97964 | 0.99649 |  0.7677 | 0.0059279 | 0.0063954 |  0.018375 | 0.080587 | 0.0064523 |  0.0022713 | 4.15E-05   | 0.53904 | 1 |
|  8 | 0.77982 | 1.6215 | 0.39222 | 0.98512 | 0.99825 | 0.80816 | 0.0050987 | 0.0047314 |  0.024875 | 0.089686 | 0.0079794 |  0.0024664 | 0.00014676 | 0.66975 | 1 |
|  9 | 0.83089 | 1.8199 | 0.45693 |  0.9824 |       1 | 0.77106 | 0.0060055 |  0.006564 | 0.0072447 | 0.040616 | 0.0016469 | 0.00038812 | 3.29E-05   | 0.33696 | 1 |
| 11 |  0.7459 | 1.4927 | 0.34116 | 0.98296 |       1 | 0.83088 | 0.0055665 | 0.0056395 | 0.0057679 | 0.036511 | 0.0013313 | 0.00030872 | 3.18E-05   | 0.25026 | 1 |
| 12 | 0.79606 | 1.6934 | 0.43387 | 0.98181 |       1 | 0.76985 | 0.0077992 |  0.011071 |  0.013677 | 0.057832 | 0.0033334 | 0.00081648 | 0.00013855 | 0.49751 | 1 |
+----+---------+--------+---------+---------+---------+---------+-----------+-----------+-----------+----------+-----------+------------+------------+---------+---+

I have two sets of row indices :

set1 = [1,3,5,8,9]

set2 = [2,4,7,10,10]

Note : Here, I have indicated the first row with index value 1. Length of both sets shall always be same.

What I am looking for is a fast and pythonic way to get the difference of column values for corresponding row indices, that is : difference of 1-2,3-4,5-7,8-10,9-10.

For this example, my resultant dataframe is the following:

+---+---------+--------+---------+---------+---------+---------+-----------+-----------+-----------+----------+-----------+------------+------------+---------+---+
| 1 | 0.01479 | 0.0515 |  0.0372 | 0.00383 | 0.00175 | 0.03725 | 0.0005857 | 0.0010551 |   0.02363 | 0.037474 | 0.0079885 |  0.0025243 | 0.00019997 | 0.47901 | 0 |
| 1 | 0.02925 | 0.1128 | 0.03622 | 0.00189 |       0 | 0.00885 | 0.0005805 | 0.0015142 |  0.004053 | 0.008046 | 0.0009816 | 0.00023372 |  0.0000284 | 0.14437 | 0 |
| 3 | 0.04319 | 0.1492 |  0.0524 | 0.00814 | 0.00175 | 0.05323 | 0.0023293 | 0.0053106 | 0.0169371 | 0.044347 |  0.005928 | 0.00190654 | 0.00012326 | 0.32761 | 0 |
| 3 | 0.03483 | 0.1265 | 0.02306 | 0.00059 |       0 | 0.00121 | 0.0017937 |  0.004507 | 0.0064323 | 0.017216 | 0.0016865 | 0.00042836 | 0.00010565 | 0.16055 | 0 |
| 1 | 0.05016 | 0.2007 | 0.09271 | 0.00115 |       0 | 0.06103 | 0.0022327 | 0.0054315 | 0.0079091 | 0.021321 | 0.0020021 | 0.00050776 | 0.00010675 | 0.24725 | 0 |
+---+---------+--------+---------+---------+---------+---------+-----------+-----------+-----------+----------+-----------+------------+------------+---------+---+

My resultant difference values are absolute here.

I cant apply diff(), since the row indices may not be consecutive. I am currently achieving my aim via looping through sets.

Is there a pandas trick to do this?

rj dj
  • 260
  • 1
  • 5
  • 22
  • 1
    Not really a duplicate, but have a closer look at https://stackoverflow.com/questions/25055712/pandas-every-nth-row – Sosel Nov 27 '17 at 11:32

3 Answers3

3

Use loc based indexing - 

df.loc[set1].values - df.loc[set2].values

Ensure that len(set1) is equal to len(set2). Also, keep in mind setX is a counter-intuitive name for list objects.

cs95
  • 379,657
  • 97
  • 704
  • 746
2

You need to select by data reindexing and then subtract:

df =  df.reindex(set1) - df.reindex(set2).values

loc or iloc will raise a future warning, since passing list-likes to .loc or [] with any missing label will raise KeyError in the future.

Bharath M Shetty
  • 30,075
  • 6
  • 57
  • 108
jezrael
  • 822,522
  • 95
  • 1,334
  • 1,252
0

In short, try the following:

df.iloc[::2].values - df.iloc[1::2].values

PS: Or alternatively, if (like in your question the indices follow no simple rule):

df.iloc[set1].values - df.iloc[set2].values
Sosel
  • 1,678
  • 1
  • 16
  • 31