Stack Overflow
Stack Overflow

Questions tagged [mclapply]

mclapply is a parallelized version of lapply, it returns a list of the same length as X, each element of which is the result of applying FUN to the corresponding element of X.

mclapply is a parallelized version of lapply. It returns a list of the same length as X, each element of which is the result of applying FUN to the corresponding element of X.

136 questions
48
votes
6 answers

Is there way to track progress on a mclapply?

I love the setting .progress = 'text' in plyr's llply. However, it causes my much anxiety to not know how far along an mclapply (from package multicore) is since list items are sent to various cores and then collated at the end. I've been outputting…
Maiasaura
  • 32,226
  • 27
  • 104
  • 108
31
votes
3 answers

Understanding the differences between mclapply and parLapply in R

I've recently started using parallel techniques in R for a project and have my program working on Linux systems using mclapply from the parallel package. However, I've hit a road block with my understanding of parLapply for Windows. Using mclapply I…
A_Skelton73
  • 1,150
  • 4
  • 12
  • 23
21
votes
2 answers

R mclapply vs foreach

I use mclapply for all my "embarassingly parallel" computations. I find it clean and easy to use, and when arguments mc.cores = 1 and mc.preschedule = TRUE I can insert browser() in the function inside mclapply and debug line by line just like in…
user7613376
21
votes
2 answers

Speed-up data.table group by using multiple cores and parallel programming

I have a large code and the aggregation step is the current bottleneck in terms of speed. In my code I'd like to speed-up the data grouping step to be faster. A SNOTE (simple non trivial example) of my data looks like this: library(data.table) a =…
Dnaiel
  • 7,622
  • 23
  • 67
  • 126
20
votes
3 answers

Printing from mclapply in R Studio

I am using mclapply from within RStudio and would like to have an output to the console from each process but this seems to be suppressed somehow (as mentioned for example here: Is mclapply guaranteed to return its results in order?). How could I…
obachtos
  • 977
  • 1
  • 12
  • 30
17
votes
8 answers

tm_map has parallel::mclapply error in R 3.0.1 on Mac

I am using R 3.0.1 on Platform: x86_64-apple-darwin10.8.0 (64-bit) I am trying to use tm_map from the tm library. But when I execute the this code library(tm) data('crude') tm_map(crude, stemDocument) I get this error: Warning message: In…
Dominik
  • 2,753
  • 7
  • 28
  • 32
15
votes
2 answers

mclapply returns NULL randomly

When I am using mclapply, from time to time (really randomly) it gives incorrect results. The problem is quite thoroughly described in other posts across the Internet, e.g. (http://r.789695.n4.nabble.com/Bug-in-mclapply-td4652743.html). However, no…
Kudzu
  • 203
  • 2
  • 5
14
votes
2 answers

R tm In mclapply(content(x), FUN, ...) : all scheduled cores encountered errors in user code

When I run the following codes to the penultimate line, I got Warning message: In mclapply(content(x), FUN, ...) : all scheduled cores encountered errors in user code When I run the final line, I got "Error in UseMethod(\"words\") : \n no…
Weijia
  • 139
  • 1
  • 1
  • 5
12
votes
1 answer

How can I get R's lapply (and mclapply) to restore the state of the random number generator?

R ignores setting .Random.seed inside of an lapply. Using set.seed however, works fine. Some code: # I can save the state of the RNG for a few seeds seed.list <- lapply( 1:5, function(x) { set.seed(x) …
Nathan VanHoudnos
  • 12,923
  • 2
  • 23
  • 29
11
votes
1 answer

Deprecation of multicore (mclapply) in R 3.0

I understand multicore is deprecated as of R version 2.14 and I was advised to start using the package parallel which comes built into the base of R 3.0. Going through the documentation of parallel, I found that there are mainly two functions to…
Omar Wagih
  • 8,504
  • 7
  • 59
  • 75
10
votes
3 answers

Warnings suppressed with mclapply in R

With mclapply() all issued warnings seems get suppressed: library(multicore) mclapply(1:3, function(x) warning(x)) [[1]] [1] "1" [[2]] [1] "2" [[3]] [1] "3" while lapply would give: lapply(1:3, function(x) warning(x)) [[1]] [1] "1" [[2]] [1]…
thulsadoom
  • 131
  • 5
10
votes
2 answers

Knitr: redirect chunk code output to terminal

I want to monitor some pretty lengthy parallelized computations embedded in a knitr file. The computations rely on a package I have written, and the relevant function uses mclapply from the multicore package for parallelization. This function…
Pepin_the_sleepy
  • 297
  • 1
  • 13
8
votes
2 answers

Parallel processing in R - setting seed with mclapply() vs. pbmclapply()

I'm parallelizing simulations in R (using mclapply() from the parallel package) and wanted to track my progress with each function call. So I instead decided to use pbmclapply() from the pbmcapply package in order to have a progress bar each time I…
bob
  • 610
  • 5
  • 23
8
votes
4 answers

why this error appears "all scheduled cores encountered errors in user code" is it related to core processor of servers?

We are analyzing sequencing data while filtering and trimming fastq files encountered following error. Is the following error due to unavailability of core for processing commands? Error in colnames<-(*tmp*, value = c("cs103_R1_dada.fastq",…
rahul
  • 81
  • 1
  • 1
  • 3
8
votes
2 answers

Parallel processing of big rasters in R (windows)

I'm using the doSNOW package and more specifically the parLapply function to perform reclassification (and subsequently other operations) on a list of big raster datasets (OS: Windows x64). The code looks a little like this minimalistic…
Val
  • 6,585
  • 5
  • 22
  • 52
1
2 3
…
9 10