0

I am having problem downloading a csv file from internet.

I tried the following code, but I wasn't able to make it work. 

myurl <- "http://unstats.un.org/unsd/mdg/Handlers/ExportHandler.ashx?Type=Csv&Series=761"
download.file(myurl, destfile="./test.csv",method="curl")

Instead I received the following output.

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0100 16313  100 16313    0     0  11961      0  0:00:01  0:00:01 --:--:-- 11959100 16313  100 16313    0     0  11960      0  0:00:01  0:00:01 --:--:-- 11959

When I manually download, a file like this is generated "MDG_Export_20150821_224828123.csv". The server generates a new file name (MDG_Export + date + time).

Is there anyway I can download this from my R program because the manual download is not an option in my project?

Thank you for your help.

scribbles
  • 4,089
  • 7
  • 22
  • 29
Adjeiinfo
  • 159
  • 1
  • 2
  • 15
  • Have you tried just reading it with `read.csv(url(myurl), ....)`? Do you really want to save it or just use it in R? – mlt Aug 22 '15 at 03:50
  • 1
    The file is a `zip` file , so you need to download it, `unzip` it then read in to R. There is also some noise at the bottom of the text file - `fread` from `data.table` package is good for making helpful assumptions. This seems to work `download.file(myurl, temp <- tempfile()) ; lst <- unzip(temp, list=TRUE) ; dat <- data.table::fread(unzip(temp, lst$Name), header=TRUE)`. See http://stackoverflow.com/questions/23899525/using-r-to-download-zipped-data-file-extract-and-import-csv and for additional links – user20650 Aug 22 '15 at 03:54
  • 1
    There is an `unz` file connection function. – IRTFM Aug 22 '15 at 05:44
  • Thank you very much. This worked for me. As you mentioned there is noise at the bottom. I had to remove it before saving the file. – Adjeiinfo Aug 22 '15 at 07:13

1 Answers1

2

Try this:

myurl <- "http://unstats.un.org/unsd/mdg/Handlers/ExportHandler.ashx?Type=Csv&Series=761&filename=MDG_Export_20150821_224828123.zip"
myzip <- unz(myurl, filename='MDG_Export_20150821_224828123.zip')
download.file(myurl, destfile="./test.zip",method="curl")

You will then need to expand the file with unzip

Martin Morgan
  • 45,935
  • 7
  • 84
  • 112
IRTFM
  • 258,963
  • 21
  • 364
  • 487