Is there a faster alternative to R 'apply' function on rows?

Asked Apr 29 '19 at 07:56

Active Apr 29 '19 at 07:56

Viewed 61 times

I would like to calculate the standard deviation for each row on a subset of columns. I am using a data table of more than 1 700 000 rows and the function 'apply'.

The line code below takes time :

apply(as.matrix(data[ , .SD, .SDcols = subset_of_columns]), 1, sd, na.rm = TRUE)

Is there an alternative ?

Thank you.

asked Apr 29 '19 at 07:56

Carozette

You could maybe take a look into parallelization http://dept.stat.lsa.umich.edu/~jerrick/courses/stat701/notes/parallel.html – boski Apr 29 '19 at 08:00
2

`rowSds` from `matrixStats` – minem Apr 29 '19 at 08:01
This works fine : system time of 0.014 instead of 0.270) ! Thank you. – Carozette Apr 29 '19 at 08:27
Thank you for that top @minem, that is awesome! – DSGym Apr 29 '19 at 08:28
`melt` the data.table and use something like `DT[, sd(values), by = rowid]`. – Roland Apr 29 '19 at 08:44

Is there a faster alternative to R 'apply' function on rows?

0 Answers0