2

Is there a way to find the modified date/time for files on an FTP server in R? I have found a great way to list all of the files that are available, but I only want to download ones that have been updated since my last check. I tried using:

info<-file.info(url)

However, it returns a pretty ugly list of nothing. My url is made up of: "ftp://username:password@FTPServer//filepath.xml"

Adi Bradfield
  • 2,063
  • 1
  • 17
  • 27
user3267547
  • 37
  • 1
  • 6

1 Answers1

4

Until we see the output from this particular FTP server (they are all different) for directory listings, here's a path you can follow:

library(curl)
library(stringr)

Get the raw directory listing:

con <- curl("ftp://ftp.FreeBSD.org/pub/FreeBSD/")
dat <- readLines(con)
close(con)
dat

## [1] "-rw-rw-r--    1 ftp      ftp          4259 May 07 16:18 README.TXT" 
## [2] "-rw-rw-r--    1 ftp      ftp            35 Sep 09 21:00 TIMESTAMP"  
## [3] "drwxrwxr-x    9 ftp      ftp            11 Sep 09 21:00 development"
## [4] "-rw-r--r--    1 ftp      ftp          2566 Sep 09 10:00 dir.sizes"  
## [5] "drwxrwxr-x   28 ftp      ftp            52 Aug 23 10:44 doc"        
## [6] "drwxrwxr-x    5 ftp      ftp             5 Aug 05 04:16 ports"      
## [7] "drwxrwxr-x   10 ftp      ftp            12 Sep 09 21:00 releases"

Filter out the directories:

no_dirs <- grep("^d", dat, value=TRUE, invert=TRUE)
no_dirs

## [1] "-rw-rw-r--    1 ftp      ftp          4259 May 07 16:18 README.TXT"
## [2] "-rw-rw-r--    1 ftp      ftp            35 Sep 09 21:00 TIMESTAMP" 
## [3] "-rw-r--r--    1 ftp      ftp          2566 Sep 09 10:00 dir.sizes"

Extract just the timestamp and filename:

date_and_name <- sub("^[[:alnum:][:punct:][:blank:]]{43}", "", no_dirs)
date_ane_name
## [1] "May 07 16:18 README.TXT"
## [2] "Sep 09 21:00 TIMESTAMP" 
## [3] "Sep 09 10:00 dir.sizes"

Put them into a data.frame:

do.call(rbind.data.frame, 
        lapply(str_match_all(date_and_name, "([[:alnum:] :]{12}) (.*)$"), 
               function(x) {
                 data.frame(timestamp=x[2],
                            filename=x[3], 
                            stringsAsFactors=FALSE)
})) -> dat
dat

##      timestamp   filename
## 1 May 07 16:18 README.TXT
## 2 Sep 09 21:00  TIMESTAMP
## 3 Sep 09 10:00  dir.sizes

You still need to convert the timestamp to a POSIXct but that's trivial.

This particular example is dependent on that system's FTP directory listing response. Just change the regexes for yours.

hrbrmstr
  • 77,368
  • 11
  • 139
  • 205