1

I have R 4.20+, so I believe utils::download.file is using capability libcurl.

I can get the headers of a url with base::curlGetHeaders(url).

Is there a parameter I can pass in download.file to return the headers, so I can't get them in the same call. Under the hood, download.file is processing the header somehow, as it is receiving it.

How to return response headers I get with curlGetHeaders(url) from the function download.file?

I am aware of external packages (e.g., Rcurl) but for the download to occur, the headers have to be received within R:::base.

Update

Here is the source code from R

"libcurl" = {
           headers <- if(length(headers)) paste0(nh, ": ", headers)
           status <- .Internal(curlDownload(url, destfile, quiet, mode, cacheOK,
                        headers))
       },

The function curlDownload has traditional curl options here (libcurl.c):

curl_easy_setopt(hnd[i], CURLOPT_HTTPHEADER, headers);

That sets the header, not return it. Why are the raw curl functions not publicly exposed. C exposes them as does PHP... see

Can PHP cURL retrieve response headers AND body in a single request?

$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
// ...

$response = curl_exec($ch);

So I guess curlDownload needs:

curl_easy_setopt(hnd[i], CURLOPT_HEADER, 1);

library (curl)

In this library, under the hood, the same syntax is being used. How to expose the syntax directly to me? From download.c:

curl_easy_setopt(handle, CURLOPT_URL, NULL);
  curl_easy_setopt(handle, CURLOPT_NOPROGRESS, 1);
  curl_easy_setopt(handle, CURLOPT_WRITEFUNCTION, NULL);
  curl_easy_setopt(handle, CURLOPT_WRITEDATA, NULL);
Dharman
  • 30,962
  • 25
  • 85
  • 135
mshaffer
  • 959
  • 1
  • 9
  • 19

1 Answers1

0

This doc mentions the handle_data function, within the curl package.

https://cran.r-project.org/web/packages/curl/curl.pdf

For example, suppose you need to peek at the status code, content type or other headers before reading all the lines. Once you have a working connection, many other functions in R will accept that ... you can use the various read.csv(x), read.delim(x), scan(x) and so on.

library('curl');
h <- new_handle();
x <- curl('https://cran.r-project.org', handle=h);
open(x);
handle_data(h)$status_code;
handle_data(h)$type;
parse_headers(handle_data(h)$headers);
y <- readLines(x);
close(x);
Tel
  • 1