Stack Overflow
Stack Overflow

Questions tagged [dompi]

17 questions
2
votes
1 answer

What's the difference between using the "doParallel" package with type = MPI and using doMPI directly?

What's the difference between using the "doParallel" package with type = MPI and using doMPI directly? library(foreach) library(doParallel) cl <- makeCluster(mpi.universe.size(), type='MPI') registerDoParallel(cl) system.time(foreach(i = 1:3)…
correocont
  • 69
  • 6
2
votes
0 answers

Open MPI error with doMPI

I am having an issue running doMPI on our Linux clusters. I am trying to run the sincMPI.R example problem in the doMPI package with an mpirun -n 1 Rscript sincMPI.R command. The error I am getting is: Open MPI does not support recursive calls of…
amelcher
  • 61
  • 1
  • 7
2
votes
1 answer

Getting mpirun to recognize all cores on each node

I have set up two nodes for MPI, aml1 (master) and aml2 (worker). I am trying to use mpirun with R scripts and using Rmpi and doMPI libraries. The specs for both machines are the same: On RHEL 7.3 # lscpu Architecture: x86_64 CPU…
Daren Eiri
  • 137
  • 12
2
votes
1 answer

doMPI and Clusters

I'm having my first experience using R and clusters. Searching on Google, I discovered the package "doMPI". This package allow me to automatizate the creation of processes. However, I'm little confuse on the functions. 1 - What is the difference…
olegario
  • 711
  • 2
  • 16
  • 35
1
vote
1 answer

R - How to iterate each slice of 4D matrix

I'm using doMPI in R to parallelize saving netCDF climate data. This data is stored in R in a 4-dimensional matrix m, with data for 6 variables, at 20000 timepoints, over a latitude and longitude grid. m is thus indexed as m[lon,lat,time,variable].…
Jamie S
  • 111
  • 1
  • 10
1
vote
1 answer

doMPI and nodes, processors and cores

I want to do a distributed parallel computation with a cluster I have access to: 5 nodes ("computers"); each node has two processors ("CPUs"), and each processor has 18 cores. So, the number of threads I could use, in an embarrassingly parallel…
Anarcocapitalista Socialdemocr
  • 67
  • 5
1
vote
1 answer

R package containing foreach will work with doParallel but not doMPI, can not find object

I'm trying to write an R package which contains several nested functions, within a foreach statement and doMPI backend. It is throwing a "cannot find "XXX" object error. The strange thing is that this error does not occur if I use doParallel as…
Jim Maas
  • 1,481
  • 2
  • 16
  • 36
1
vote
1 answer

Write to log file using doMPI

I am running doMPI on an HPC and I would like to log output from workers. Using doParallel, I was able to use makeCluster(outfile='myfile.log'). With doMPI, there does not seem to be an outfile argument in any of the methods. I tried using…
Ben Carlson
  • 1,053
  • 2
  • 10
  • 18
1
vote
1 answer

doMPI error: There are not enough slots available in the system to satisfy the 2 slots that were requested by the application

I'm trying to get the doMPI package running on my local machine, so that I can perform testing on it before submitting jobs to a cluster. I am using Mac OSX Yosemite and have installed open mpi 2.0.2 via brew. mpirun -V mpirun (Open MPI)…
Ben Carlson
  • 1,053
  • 2
  • 10
  • 18
1
vote
1 answer

doMPI not recognizing other nodes in cluster for R script

Using RHEL7.3 Using R 3.3.2 Installed Rmpi_0.6-6.tar.gz and doMPI_0.2.1.tar.gz Installed mpich-3.0-3.0.4-10.el7 RPM for x86_64 I created a cluster of three machines (aml1,2,3). I can run the /examples/cpi example from the mpich installation and…
Daren Eiri
  • 137
  • 12
1
vote
0 answers

R, foreach, doMPI reduced speed with increased iterations

I've used the combination of R, doMPI, and foreach on a cluster for several years now and usually increasing simulation iteration numbers is pretty linear, in terms of execution time required. Recently I've been using this nested foreach loop, and…
Jim Maas
  • 1,481
  • 2
  • 16
  • 36
0
votes
0 answers

Details of foreach + doMPI: Multiple foreach loops in sequence in the same script?

In R, I am using the package foreach with doMPI in a wrapper script run an external model many times in parallel on a cluster. Each MPI process gets one parameter point for which to execute the model. However, to run this, there's also a bit of pre-…
Jørgen Eriksson Midtbø
  • 1
  • 2
0
votes
1 answer

Rmpi on Ubuntu 21.04 Desktop with MPICH Virgin Build DoMPI

I have a virgin build of 64-bit Ubuntu 21.04 build on a Z820 Intel Xeon(R) CPU E5-2643 v2 @ 3.50GHz × 24. The OS was installed with default settings. MPICH install: For the particular system mpich version 3.4.2 via the following steps. cd…
BabblingREnthusiast
  • 43
  • 6
0
votes
1 answer

doMPI and user-defined packages

I am starting to use doMPI, and I user a package I have defined by myself. First, I have in my file to be executed: library(doMPI) cl <- startMPIcluster() registerDoMPI(cl) Note: I am not using cl <- startMPIcluster(count), since I believe it is…
Anarcocapitalista Socialdemocr
  • 67
  • 5
0
votes
1 answer

Functions not finding variables, or environment, when inside a foreach loop

I'm attempting to build an R package that contains a top level function inside a couple of nested foreach loops. This top level function then calls a set of further nested functions. The problem I've got is of lexical scoping, the lower level…
Jim Maas
  • 1,481
  • 2
  • 16
  • 36
1
2