2

I got this error "Cannot allocate vector in R of size 11.8 Gb" because my desktop has 8 gb ram and size of matrix what I used is 3000x52128.

In here is any solution to avoid this memory error? Even though I tried to decrease size of matrix as 1500x52128, also I got same error and same size as 11.8 Gb.

So what should I do? Only one solution is to work on the computer with 16 gb ram?

Addtional comment:
when following commands are running, I got this error.

svmDS <- read.csv("TrainDataSet_ver1.2.csv");
model<-naiveBayes(as.factor(class)~., data=svmDS)
Khuyagbaatar Batsuren
  • 165
  • 2
  • 2
  • 16
  • Well, consider carefully if you need to do the operation that would create an 11.8 GB vector. Your matrix isn't really big, so there is a chance that your code does something you didn't expect or could implement in a different (more momory efficient) way. – Roland Jan 27 '14 at 12:31
  • 1
    That matrix should be about 1.2GB assuming 8 byte values (maybe it's text?). Your code is doing something else, as clearly indicated by the smaller matrix not making a smaller memory allocation. You need to post more details if you want a good answer. – John Jan 27 '14 at 12:34
  • I'm really sorry guys. I added the comment – Khuyagbaatar Batsuren Jan 27 '14 at 12:51
  • It's worth noting that since R is a community project, you can crack open the `naiveBayes` function and experiment with tuning it for your purpose. There are R libraries that allow you to work with data outside of RAM (on disk), for example, so you can write a function that suits your environment if you need. If it's a major improvement, you can even submit the changes to the package maintainer. – bright-star Jun 16 '14 at 06:36

1 Answers1

4

The below function is helpful to free the workspace , by removing large objects which you already have in the workspace. This is not a direct solution to your problem. But it also helps.

.ls.objects <- function (pos = 1, pattern, order.by,
                        decreasing=FALSE, head=FALSE, n=5) {
    napply <- function(names, fn) sapply(names, function(x)
                                         fn(get(x, pos = pos)))
    names <- ls(pos = pos, pattern = pattern)
    obj.class <- napply(names, function(x) as.character(class(x))[1])
    obj.mode <- napply(names, mode)
    obj.type <- ifelse(is.na(obj.class), obj.mode, obj.class)
    obj.prettysize <- napply(names, function(x) {
                           capture.output(print(object.size(x), units = "auto")) })
    obj.size <- napply(names, object.size)
    obj.dim <- t(napply(names, function(x)
                        as.numeric(dim(x))[1:2]))
    vec <- is.na(obj.dim)[, 1] & (obj.type != "function")
    obj.dim[vec, 1] <- napply(names, length)[vec]
    out <- data.frame(obj.type, obj.size, obj.prettysize, obj.dim)
    names(out) <- c("Type", "Size", "PrettySize", "Rows", "Columns")
    if (!missing(order.by))
        out <- out[order(out[[order.by]], decreasing=decreasing), ]
    if (head)
        out <- head(out, n)
    out
}

lsos <- function(..., n=10) {
    .ls.objects(..., order.by="Size", decreasing=TRUE, head=TRUE, n=n)
}

lsos()

Which would show you the list of the objects in your workspace and to occassionally rm() some of them.

Prasanna Nandakumar
  • 4,295
  • 34
  • 63