sftp get files with R

Question

I need to sftp get bunch of files and and parse them and get rid of unwanted lines and form a data frame. Is it possible to sftp get bunch of files from remote ftp server and process these txt files in R?

This is what I have tried so far but I get authentication error:

library(RCurl) 
url="sftp://ftp.address.com/directory"
filenames = getURL(url, ftp.use.epsv = FALSE, ftplistonly = TRUE)
filenames = paste(url, strsplit(filenames, "\n")[[1]], sep = "")
con = getCurlHandle( ftp.use.epsv = FALSE)
sapply(filenames, getURL, curl = con)

I get this error:

Error in function (type, msg, asError = TRUE) : Could not resolve host: sftp:; No data record of requested type

ok, I have done this and it is sort of working.

getURL("sftp://site.com/filename.txt", userpwd="id:passed")

I am not done yet. I need the output of getURL to be written to a file so that I can do readLines on the file to parse the unwanted lines and etc.

Yes. Yes, it is. [What have you tried?](http://whathaveyoutried.com) — , Apr 11 '13 at 13:45
@Jack Maney, I am trying to use RCurl but I dont see any good examle with sftp. Onlything I see it http. — user1471980, Apr 11 '13 at 14:06
@themel, where in that code I put username and password. Seriouly, I couldn't come accross any good example of this. — user1471980, Apr 11 '13 at 14:22
e.g. http://stackoverflow.com/questions/10157969/rcurl-http-authentication-when-site-responds-with-http-401-code-without-www-aut — themel, Apr 11 '13 at 14:41
@themel, that link is for https. I am tryin to get files from the sftp server. — user1471980, Apr 11 '13 at 15:08
You are right, obviously the userpwd option must be useless if the protocol is not HTTPS... m( — themel, Apr 11 '13 at 15:36

score 6 · Accepted Answer · answered Apr 11 '13 at 15:41

6

x<-getURL("sftp://site.com/filename.txt", userpwd="id:passed")
fileConn<-file("output.txt")
writeLines(x, fileConn)
close(fileConn)
y<-readLines("output.txt")

now I can parse out the y. This is what I come up with. Let me know there may be a better way to do this?

answered Apr 11 '13 at 15:41

user1471980

10,127
48
136
235

1

Any idea on how to do this with other file types? Such as sas7bdat files – Justin Klevs May 08 '17 at 20:28
@Justin Klevs looks like those are binary files. You should try using `getURLContent`, which is a high-level function that automatically determines whether the content is text or binary. Or, if you're sure you'll be downloading only binary data, then go with `getBinaryURL` . – mbiron May 15 '18 at 14:27
1

`Protocol “sftp” not supported or disabled in libcurl` We have problems with sftp not being supported. Did any of you also had problems with that? – Rvanlaak Nov 01 '18 at 11:50

sftp get files with R

1 Answers1

Linked

Related