Stack Overflow
Stack Overflow

Questions tagged [httr]

The R package 'httr' provides useful tools for working with HTTP connections. It is a simplified wrapper built on top of 'RCurl'.

The R package httr provides useful tools for working with HTTP connections. It is a simplified wrapper built on top of RCurl. It is much much less configurable but because it only attempts to encompass the most common operations it is also much much simpler.

Resources:

1019 questions
36
votes
1 answer

Upload a file over 2.15 GB in R

I've got a manual process where I'm uploading 5-6 GB file to a web server via curl: curl -X POST --data-binary @myfile.csv http://myserver::port/path/to/api This process works fine, but I'd love to automate it using R. The problem is, I either…
Zach
  • 29,791
  • 35
  • 142
  • 201
27
votes
4 answers

How to download a file behind a semi-broken javascript asp function with R

I am trying to fix a download automation script that I provide publicly so that anyone can easily download the world values survey with R. On this web page - http://www.worldvaluessurvey.org/WVSDocumentationWV4.jsp - the PDF link…
Anthony Damico
  • 5,779
  • 7
  • 46
  • 77
20
votes
1 answer

Send a POST request using httr R package

I can't figure out how to imitate what the browser does, when sending the server data via a POST request. Here are the relevant URLs with explanation below. (1) http://kenpom.com/ (2) http://kenpom.com/register.php?frompage=1
panstotts
  • 623
  • 1
  • 5
  • 13
19
votes
2 answers

Scrape password-protected website in R

I'm trying to scrape data from a password-protected website in R. Reading around, it seems that the httr and RCurl packages are the best options for scraping with password authentication (I've also looked into the XML package). The website I'm…
itpetersen
  • 1,475
  • 3
  • 13
  • 32
19
votes
2 answers

automating the login to the uk data service website in R with RCurl or httr

I am in the process of writing a collection of freely-downloadable R scripts for http://asdfree.com/ to help people analyze the complex sample survey data hosted by the UK data service. In addition to providing lots of statistics tutorials for these…
Anthony Damico
  • 5,779
  • 7
  • 46
  • 77
16
votes
4 answers

httr github-API callback URL issues

I am using the httr v0.2 package to use the github api as well now. But I am struggling to get past the oauth2.0(...) part in which I get to the browser page for my app, click on 'Allow' and then get redirected to the callback URL page. The httr…
h.l.m
  • 13,015
  • 22
  • 82
  • 169
15
votes
1 answer

Using oauth2.0 tokens with R's httr package

Problem The package httr provides CURL wrappers in R (see package documentation). I'm brand new to HTTP and APIs. My trouble is getting oauth2.0 authentication to work. I have tried various syntax and get either errors or status 401. What is the…
Danielle
  • 733
  • 1
  • 10
  • 24
15
votes
1 answer

Using R to "click" a download file button on a webpage

I am attempting to use this webpage http://volcano.si.edu/search_eruption.cfm to scrape data. There are two drop-down boxes that ask for filters of the data. I do not need filtered data, so I leave those blank and continue on to the next page by…
abet
  • 175
  • 1
  • 8
15
votes
1 answer

Multi POST query (session mode)

I am trying to interrogate this site to get the list of offers. The problem is that we need to fill 2 forms (2 POST queries) before receiving the final result. This what I have done so far: First I am sending the first POST after setting the…
agstudy
  • 119,832
  • 17
  • 199
  • 261
15
votes
3 answers

"You must provide a hash." error when using API to download data (in R)

I would like to extract the data from MARVEL DEVELOPER by API code and analyze it (using R). I got the following url from MARVEL website: http://gateway.marvel.com:80/v1/public/characters?apikey=f389fcb49ad574e10ca570867f4bfa43 I used httr package…
Tal Reznik
  • 151
  • 1
  • 1
  • 3
14
votes
2 answers

Redirect in Shiny app

I'm trying to make my Shiny app to redirect the user to another page. I'm using httr to send GET requests and see if the user is logged in. If he's not, I want to redirect him to another link. Can I do that using R / Shiny only, or do I need some…
vladli
  • 1,454
  • 2
  • 16
  • 40
14
votes
3 answers

R: Check existence of url, problems with httr:GET() and url.exists()

I have a list of about 13,000 URLs that I want to extract info from, however, not every URL actually exists. In fact the majority don't. I have just tried passing all 13,000 urls through html() but it takes a long time. I am trying to work out how…
Adam
  • 1,147
  • 3
  • 15
  • 23
14
votes
2 answers

How can I screenshot a website using R?

So I'm not 100% sure this is possible, but I found a good solution in Ruby and in python, so I was wondering if something similar might work in R. Basically, given a URL, I want to render that URL, take a screenshot of the rendering as a .png, and…
Zach
  • 29,791
  • 35
  • 142
  • 201
14
votes
1 answer

How to login and then download a file from aspx web pages with R

I'm trying to automate the download of the Panel Study of Income Dynamics files available on this web page using R. Clicking on any of those files takes the user through to this login/authentication page. After authentication, it's easy to…
Anthony Damico
  • 5,779
  • 7
  • 46
  • 77
13
votes
1 answer

SOAP request in R

Does anyone know how to formulate following SOAP request with R? POST /API/v201010/AdvertiserService.asmx HTTP/1.1 Host: advertising.criteo.com Content-Type: text/xml; charset=utf-8 Content-Length: length SOAPAction:…
jburkhardt
  • 675
  • 1
  • 4
  • 18
1
2 3
…
67 68