downloading files from the secure ftp server with R

Question

I need to download files from sftp server and parse them and insert to contents to the database.

I am currently using rCurl as follows:

library(RCurl) 
url<-c("sftp://data.ftp.net/incomining.data.txt")
x<-getURL(url, userpwd="<id>:<passwd>")
writeLines(x, incoming.data.txt"))

I also looked at download.file and I dont see sftp sufpport in download.file. Has anybody else done similiar work like this? Since I will be getting multiple files, I noticed that rcurl sometimes times out. I like to sftp download all the files from the sftp server first then process it. Any ideas?

score 2 · Accepted Answer · edited May 23 '17 at 11:45

2

It sounds like the question is "how do I avoid timeouts in rcurl?"

Increase the value of CURLOPT_CONNECTTIMEOUT. This is really just the same problem as Setting Curl's Timeout in PHP .

Edit, from comments below:

x<-getURL(url, userpwd="<id>:<passwd>", connecttimeout=60) // 60 seconds, e.g.

edited May 23 '17 at 11:45

Community

1
1

answered Apr 18 '13 at 18:03

Camille Goudeseune

2,934
2
35
56

do you know where to set the RCurl timeout in R? – user1471980 Apr 18 '13 at 20:29
@user1471980 check out `curlOptions()` – Brian Diggs Apr 18 '13 at 20:43
For 60 seconds, http://www.omegahat.org/RCurl/philosophy.html suggests appending **connecttimeout=60**, like this: `x<-getURL(url, userpwd=":", connecttimeout=60)` – Camille Goudeseune Apr 18 '13 at 23:06

downloading files from the secure ftp server with R

1 Answers1

Linked