3

Background

I have an R script that runs on a host, and starts jobs on a server. The jobs output large hdf5 files.

The rest of my work is done on the host. However, the R hdf5 libraries only work on the server and not the host (because of a known bug).

From R on the host, can I connect to an R session on the server, load the R hdf5 libraries, extract data from an hdf5 file, and then pass this object to the host?

Current approach

My current kludge is to call a bash script from within R that invokes an R script on the server and does the extraction to Rdata files, and then use rsync to bring it back to the host (all within system())

Question

Is there a better way to extract data from these hdf5 files on the server?

I am thinking of something like entering a server R session in the same way that I can enter the browser session when debugging.

Community
  • 1
  • 1
David LeBauer
  • 31,011
  • 31
  • 115
  • 189
  • Do the HDF5 bindings work in other languages? If so, maybe use the h5py and rpy2 packages in Python to ship the data over to R. – kenm Mar 26 '11 at 00:56
  • @kwatford h5utils do work, although the R utils are easier (for me) to use and I would prefer not to have to rewrite the code. – David LeBauer Mar 26 '11 at 00:58

1 Answers1

2

How about Rserve, or there is a video of using svSocket for passing data between client and server on the data.table homepage.

Matt Dowle
  • 58,872
  • 22
  • 166
  • 224