2

I am trying to make a file connection within a cluster (using parallel). While it works correctly in the global environment, it gives me an error message when used within the members of the cluster (See the script below). Do I missed something?

Any suggestion?

Thanks,

# This part works
#----------------
cat("This is a test file" , file={f <- tempfile()})
con <- file(f, "rt")


# Doing what I think is the same thing gives an error message when executed in parallel
#--------------------------------------------------------------------------------------

library(parallel)
cl <- makeCluster(2)

## Exporting the object f into the cluster

clusterExport(cl, "f")
clusterEvalQ(cl[1], con <- file(f[[1]], "rt"))
 #Error in checkForRemoteErrors(lapply(cl, recvResult)) :
 # one node produced an error: cannot open the connection


## Creating the object f into the cluster

clusterEvalQ(cl[1],cat("This is a test file" , file={f <- tempfile()}))
clusterEvalQ(cl[1],con <- file(f, "rt"))
 #Error in checkForRemoteErrors(lapply(cl, recvResult)) :
 # one node produced an error: cannot open the connection 


############ Here is my sessionInfo() ###################
# R version 3.3.0 (2016-05-03)
# Platform: x86_64-w64-mingw32/x64 (64-bit)
# Running under: Windows 7 x64 (build 7601) Service Pack 1
#
# locale:
# [1] LC_COLLATE=French_Canada.1252  LC_CTYPE=French_Canada.1252   
# [3] LC_MONETARY=French_Canada.1252 LC_NUMERIC=C                  
# [5] LC_TIME=French_Canada.1252    
#
# attached base packages:
# [1] stats     graphics  grDevices utils     datasets  methods   base 
#
Arnaud
  • 377
  • 1
  • 2
  • 11
  • 1
    Just for the record, The problem came from a security feature of the McAfee antivirus avoiding the execution of a script using the windows temporary folder and other folders like C:\temp. So the simple solution is to create this file into another folder (I suppose a folder without "temp" in its name). – Arnaud Jul 08 '16 at 19:09

1 Answers1

1

Try changing the code to return a NULL rather than the created connection object:

clusterEvalQ(cl[1], {con <- file(f[[1]], "rt"); NULL})

Connection objects can't be safely sent between the master and workers, but this method avoids that.

Steve Weston
  • 19,197
  • 4
  • 59
  • 75
  • Thanks for the answer, but unfortunatly it does not work! The problem seems related to the configuration of the security on the computer I use at work. I tried the same script on my personal computer and it worked well (even without the NULL part). – Arnaud May 31 '16 at 16:03
  • @Arnaud Is your work computer running Windows? I've heard that the workers don't always have permission to read files created by `tempfile` on Windows. You could try creating the files in your home directory without using `tempfile` to see if that makes a difference. I've never run into that kind of permission problem on OS X or Linux. – Steve Weston May 31 '16 at 16:32
  • I added my sessionInfo() to my original request. – Arnaud Jun 02 '16 at 14:32
  • Thanks for the suggestion ! I will test that. – Arnaud Jun 02 '16 at 14:33
  • Both my work computer and my personal computer run win7 with the same version of R. I tried on other computers at work using previous version of R but I have the same problem. It seems related to a recent windows security update at work (the same code worked before on the same computers). If someone can give me a clue on the problem, I could be more specific when asking the IT to resolve that (if possible). – Arnaud Jun 02 '16 at 14:38
  • Finally you had the solution, the problem was linked to the windows temp folder (see my comment below my question). When I tried your suggestion, I tried to create the file into the C;\temp folder but this does not work ... however using C:\test\filename.csv works !! Thanks for your help – Arnaud Jul 08 '16 at 19:12