0

I am having trouble downloading data from an external ftp-source for use in a shinyapp. http://Shinyapps.io does not allow for writing to a file on the server, so we must avoid that and try saving externally with ftp and then later read it directly into memory from there.

See Wes Sauder at https://stackoverflow.com/a/32631356/3493503 for where I got started.

library(RCurl)
a <- "test"
saveRDS(a, file="data.Rdata")

data_path <- my_data_path
url <- my_ftp_url
ftpUpload(data_path, to=url, connecttimeout=120) #upload

bin = getBinaryURL(url, verbose = TRUE,
               ftp.use.epsv = TRUE)
readRDS(rawConnection(bin))

Uploading works fine, I can download the file with Filezilla afterwards and read it with readRDS. However, trying to download it in R with the above code does not work:

 > library(RCurl)
    > a <- "test"
    > saveRDS(a, file="data.Rdata")
    > data_path <- "**************"
    > url <-"ftp://******************/results.RData"
    > ftpUpload(data_path, to=url, connecttimeout=120) #upload
    OK 
    0 
    > bin = getBinaryURL(url, verbose = TRUE,
    + ftp.use.epsv = TRUE)
    *   Trying ***.***.***.***...
    * Connected to ******************** port 21 (#0)
    < 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
    < 220-You are user number 9 of 50 allowed.
    < 220-Local time is now 10:43. Server port: 21.
    < 220-This is a private system - No anonymous login
    < 220-IPv6 connections are also welcome on this server.
    < 220 You will be disconnected after 15 minutes of inactivity.
    > USER ********
    < 331 User ******** OK. Password required
    > PASS ********
    < 230 OK. Current restricted directory is /
    > PWD
    < 257 "/" is your current location
    * Entry path is '/'
    > EPSV
    * Connect data stream passively
    * ftp_perform ends with SECONDARY: 0
    < 229 Extended Passive mode OK (|||31331|)
    *   Trying ********...
    * Connecting to ******** port 31331
    * Connected to ******** port 21 (#0)
    > TYPE I
    < 200 TYPE is now 8-bit binary
    > SIZE results.RData
    < 213 50
    > RETR results.RData
    < 150 Accepted data connection
    * Maxdownload = -1
    * Getting file with size: 50
    * Remembering we are in dir ""
    < 226-File successfully transferred
    < 226 0.000 seconds (measured here), 0.66 Mbytes per second
    * Connection #0 to host ******** left intact
    > readRDS(rawConnection(bin))
    Error in readRDS(rawConnection(bin)) : unknown input format

For the matter of completeness, I also tried

load(rawConnection(bin))
Error in load(rawConnection(bin)) : 
the input does not start with a magic number compatible with loading from a     connection

source(rawConnection(bin))
Error in source(rawConnection(bin)) : 
rawConnection(bin):1:1: unexpected input
1: 
   ^

Any solution/explanation/input is greatly appreciated.

I uploaded a file via Filezilla containing the RDS-object to http://webkartet.no/data.Rdata for you to try. It causes the same error.

Note: I tried to follow up in the comment-section of Wes Sauders answer, but was not able to do so, apparently due to a lack of SE-points. Tried to find solutions on SO without success.

Other information

platform x86_64-w64-mingw32
arch x86_64
os mingw32
system x86_64, mingw32
status
major 3
minor 3.1
year 2016
month 06
day 21
svn rev 70800
language R
version.string R version 3.3.1 (2016-06-21) nickname Bug in Your Hair

Using Windows 7

Community
  • 1
  • 1
  • make sure you're uploading in binary mode. I can't test this since I can't encourage use of such an insecure protocol. – hrbrmstr Oct 26 '16 at 10:32
  • Thank you for your comment. My understanding was that saveRDS saves in binary by default. I am unsure of what parameters to set in ftpUpload, if any. You set me on the right track though, as downloading by getURL() does work fine. Can you elaborate on the insecurity of the protocol? Are name/password sent in 'broad daylight'? What alternative options should I consider? Thanks for help. – user3493503 Oct 26 '16 at 12:30
  • Ideally, you'd be using an SFTP server or HTTPS server. As it stands, all your credentials are in the clear. I need to re-check RCurl options to see what params to use to force binary upload. – hrbrmstr Oct 26 '16 at 13:31

0 Answers0