10

First of all, this may need to be moved to superuser. I couldn't decide which venue was better.

I am trying to write an R script that will run at boot/reboot and add that machine to a pool of doRedis workers. (doRedis is a foreach backend).

Here is my R script, "~/Rworker.R"

#Define Parameters
require(multicore)
Host <- 'ip_of_doRedis_Server'
cores <- multicore:::detectCores()
TO <- 24*3600

#Start Workers
require('doRedis')
startLocalWorkers(cores,'jobs',host=Host,port=6379, timeout = TO)

I can run this script from the command line, using the command sudo R CMD BATCH ~/Rworker.R ~/RLog.

Next, I wrote a shell script to run the R script, titled "/etc/init.d/StartWorkers.sh"

#!/bin/sh
sudo echo "Starting R workers"
sudo R CMD BATCH ~/Rworker.R ~/RLog

I made this shell script executable, using chmod +x StartWorkers.sh. When I run ./StartWorkers.sh everything works great and the R session starts up and the workers get added to the pool.

Now, I need this shell script to run when I boot/reboot the machine, so I type update-rc.d StartWorkers.sh defaults. This command appears to work, but I get the following warning: 'update-rc.d: warning: /etc/init.d/StartWorkers.sh missing LSB information'

However, a check with rcconf confirms that "StartWorkers.R" is on the startuplist.

However, when I reboot the machine, the script fails to run. What am I doing wrong? The shell script runs fine from the command line, but fails when I try to run it at startup.

/EDIT: ok, per Dirk's answer, I installed littler, and changed 'StartWorkers.sh' to the following:

#! /usr/bin/r

#Define Parameters
require(multicore)
Host <- 'zachec2.dyndns.org'
cores <- multicore:::detectCores()
TO <- 24*3600

#Start Workers
require('doRedis')
startLocalWorkers(cores,'jobs',host=Host,port=6379, timeout = TO)

But when I run it, I get the following output:

Loading required package: utils
Loading required package: multicore
Warning message:
In library(package, lib.loc = lib.loc, character.only = TRUE, logical.return = TRUE,  :
  there is no package called 'multicore'
Error in loadNamespace(name) : there is no package called 'multicore'
Calls: ::: ... tryCatch -> tryCatchList -> tryCatchOne -> <Anonymous>
Execution halted

I know I have multicore installed on my system!

/EDIT2: I had to move all my R packages to cd /usr/lib/R/site-library and now the littler shell script works. I added the script to /etc/rc.local and it starts up perfectly!

Zach
  • 29,791
  • 35
  • 142
  • 201
  • I'd say Superuser would in fact be better for this. – Daniel Dickison Jun 17 '11 at 19:46
  • Add a `print(.libPaths())`. Maybe you have multicore in *your* `~/R/library` which `root` does not see. One remedy: `apt-get install r-cran-multicore`. – Dirk Eddelbuettel Jun 17 '11 at 20:05
  • When I add that line, I get the following result before the error `[1] "/usr/local/lib/R/site-library" "/usr/lib/R/site-library" [3] "/usr/lib/R/library" "/usr/lib64/R/library"` – Zach Jun 17 '11 at 20:07
  • @Dirk Eddelbuettel: I ran `apt-get install r-cran-multicore`, but now littler can't find the `doRedis` package, and unfortunately `apt-get install r-cran-doRedis`doesn't work – Zach Jun 17 '11 at 20:11
  • I install all my local packages in `/usr/local/lib/R/site-library`; that way all users find them. If you packages are scattered over your system, add location options to the `library()` call. – Dirk Eddelbuettel Jun 17 '11 at 20:14
  • @Dirk Eddelbuettel: This may be a dumb question, but how do I figure out where R has been installing my packages? It turns out it's not any of the 4 directories I listed earlier, which explains my problem... – Zach Jun 17 '11 at 20:21
  • Maybe `locate doRedis` can help. – Dirk Eddelbuettel Jun 17 '11 at 20:25
  • 1
    @Zach fire up R and look at the output of `.libPaths()` it will till you where R will look for it's packages. – Gavin Simpson Jun 17 '11 at 20:26
  • @Dirk: I ran .libPaths() from an interactive R session and figured it out. Thank you. – Zach Jun 17 '11 at 20:29
  • Please do **not use `/usr/lib/R/site-library`** by hand, use `/usr/local/lib/R/site-library`. The **`local`** matters. – Dirk Eddelbuettel Jun 17 '11 at 20:56
  • @Dirk Eddelbuettel: Pardon my ignorance, but what's the difference between the 2 libraries? When I installed multicore using apt-get, ut sent it to `/usr/lib/R/site-library` so I assumed that's where packages needed to be for littler to find them. – Zach Jun 17 '11 at 21:19
  • The package management system owns `/usr` and below; you as admin own `usr/local`. They are separate for a reason, and your sanity will be greater if you do not mix them. – Dirk Eddelbuettel Jun 17 '11 at 21:21
  • @Dirk Eddelbuettel: I've already mixed them, by copying all my packages from `/usr/local/lib64/R/library` to `/usr/lib/R/site-library`. Should I move all of them (except multicore, which was previously there) to `/usr/local/lib/R/site-library`? – Zach Jun 17 '11 at 21:27
  • Here is what I would do: Find the ones which `dpkg` or `apt` installed into `/usr/lib/R/site-library` an leave them. The others you installed, move them to `/usr/local/....` – Dirk Eddelbuettel Jun 17 '11 at 21:29
  • @Dirk Eddelbuettel. Done. Thanks for the warning! – Zach Jun 17 '11 at 21:37
  • Hi! I am having the same issue you had and cant seem to get it working. When you said you added the script to /etc/rc.local you added a link to your script like ./etc/yourscript.sh or you added your code inside rc.local? Thanks a lot for any help you can provide! – JordanBelf Sep 03 '12 at 04:54

1 Answers1

8

This is a bit of an R question, and a bit on an Ubuntu sysadmining question. here are a few points:

  1. For simple start-up tasks, I recommned just using /etc/rc.local where you can append you jobs.

  2. I just don't like R CMD BATCH which is why Jeff Horner and I wrote littler which gives you /usr/bin/r and much easier R scripting. R itself also gives you Rscript; either one is preferable over R CMD BATCH.

  3. To test scripts, just run them as root. Once theyw ork, add them to /etc/rc.local.

Hope this helps. The r-sig-debian list is a good source of Ubuntu / Debian tips too.

Dirk Eddelbuettel
  • 360,940
  • 56
  • 644
  • 725
  • I have never heard of Rscript. How does that works? And I agree... I hate R CMD BATCH with a passion. – Zach Jun 17 '11 at 19:49
  • Rscript is provided by R Core and in your Ubuntu binaries as /usr/bin/Rscript so see its documentation. Our /usr/bin/r (which you can `apt-get install littler`) came out just before Rscript and is yet a little lighter and faster. Play with either and you'd be happy. – Dirk Eddelbuettel Jun 17 '11 at 19:52
  • I got the littler script to work and added the following line to `/etc/rc.local`: `sudo /etc/init.d/StartWorkers.sh`. However, the script still appears to not be running when I boot – Zach Jun 17 '11 at 20:51
  • Oh boy. First, no need for sudo as `/etc/rc.local` is executed by root itself. Second, I suggested `/etc/rc.local` *instead of* going through `/etc/init.d/`. Third, if it breaks you get to keep the pieces and debug yourself. There is nothing we can do from here given the info your provided. Sorry. – Dirk Eddelbuettel Jun 17 '11 at 20:55
  • I removed the sudo, removed the script from `/etc/init.d/` and ran it solely through `/etc/rc.local`, and it's now working perfectly. Thank you so much for all of your help, I'm very happy to discover littler. – Zach Jun 17 '11 at 21:04
  • 1
    Cool! Good to see your persistence paid off. These are nitty little details but now that you mastered this you can automate the hell out of it. – Dirk Eddelbuettel Jun 17 '11 at 21:07
  • I appreciate your patience. Thank you! – Zach Jun 17 '11 at 21:16