parLapply within R6 classes

Question

I'm looking to use parLapply() on windows within an R6 object and noticed (that in at least some cases) that I do not need to export the R6 functions or data to the nodes.

Here is an example where I can access private methods within parLapply():

require(R6);require(parallel)
square <-
R6Class("square",
        public = list(
            numbers = NA,
            squares = NA,
            initialize = function(numbers,integer) {
                self$numbers <- numbers
                squares <- private$square.numbers()
            }
        ),
        private = list(
            square = function(x) {
                return(x^2)
            },
            square.numbers = function() {
                cl <- makeCluster(detectCores())
                self$squares <- parLapply(cl,
                                          self$numbers,
                                          function (x) private$square(x)
                                          )
                stopCluster(cl)
            }
        ))
##Test
test <- square$new(list(1,2,3))
print(test$squares)
# [[1]]
# [1] 1
# 
# [[2]]
# [1] 4
# 
# [[3]]
# [1] 9

And a second example where I can also access public members:

square2 <-
R6Class("square2",
        public = list(
            numbers = NA,
            squares = NA,
            integer = NA,
            initialize = function(numbers,integer) {
                self$numbers <- numbers
                self$integer <- integer
                squares <- private$square.numbers()
            }
        ),
        private = list(
            square = function(x) {
                return(x^2)
            },
            square.numbers = function() {
                cl <- makeCluster(detectCores())
                self$squares <- parLapply(cl,
                                          self$numbers,
                                          function (x) private$square(x)+self$integer
                                          )
                stopCluster(cl)
            }
        ))
##Test
test2 <- square2$new(list(1,2,3),2)
print(test2$squares)
#[[1]]
#[1] 3
#
#[[2]]
#[1] 6
#
#[[3]]
#[1] 11

My question is twofold: (1) What about R6 makes this possible so that I don't need to export data objects and functions; and (2) can I rely on this behavior or is this an artifact of these specific examples?

UPDATE:

This behavior also appears to work using public methods and members after the object has been instantiated:

square3 <- R6Class(
    classname = "square3",
    public = list(
        numbers = NA,
        squares = NA,
        integer = NA,
        square = function(x) {
           return(x^2)
        },
        square.numbers = function() {
            cl <- makeCluster(detectCores())
            self$squares <- parLapply(cl,
                                      self$numbers,
                                   function (x) self$square(x)+self$integer
                                  )
        stopCluster(cl)
    },
    initialize = function(numbers,integer) {
        self$numbers <- numbers
        self$integer <- integer
    }
    )
)
test3.obj <- square3$new(list(1,2,3),2)
test3.obj$square.numbers()
test3.obj$squares

# [[1]] 
# [1] 3
#
# [[2]]
# [1] 6
#
# [[3]]
# [1] 11

score 2 · Accepted Answer · answered Jan 28 '16 at 16:41

2

With R6 classes, every time you instantiate an object, that object gets a copy of each function/method, with a modified environment. The functions are assigned an environment where self points to the object's public environment (this is the public face of the object), and private points to the object's private environment.

This is different from S3 methods, which don't get copied for each instantiation of an object.

In summary: with R6, everything is self-contained in the object; with S3, the object does not contain methods.

I'm not that familiar with using parLapply, but I think that it's safe to rely on things working like that with parLapply.

answered Jan 28 '16 at 16:41

wch

4,069
2
28
36

I’m a bit puzzled by this explanation: shouldn’t the same also work with S3, simply because each fork of R gets its own copy of the working set (the OS kernel handles this, not R)? At any rate, I’ve always been using `mclapply` like this (I don’t use `parLapply` or explicit clusters), I’ve never had to export anything. – Konrad Rudolph Jan 28 '16 at 16:56
It sounds like your working on os x/linux. From my (limited) understanding, on these operating systems the "fork" gets the entire working directory, which doesn't happen on windows. That's why i was so curious about this behavior as it won't require us windows users to export any methods or members (and thus making the parallel implementation much more convenient). Also, i haven't worked much with s3, so I can't comment on its behavior in a similar situation – chandler Jan 28 '16 at 17:07

parLapply within R6 classes

1 Answers1