0

You can use multithreading in Python and send lots of http requests, like in this SO question. My question is, is there any easy way to do this in R? I've seen a guide for RCurl here, but I'd prefer a simpler solution if possible. Currently I'm looping through a series of ids, it's be great to send all (or more) of them at once.

Community
  • 1
  • 1
RobertMyles
  • 2,673
  • 3
  • 30
  • 45
  • There are a few ways to multi thread in R. I find the easiest way involves turning a chunk of code into a function, then calling it with something like `plyr` after registering a parallel backend. If you show a minimal example of your code, I'm sure we can give more specific help. – rosscova Jan 20 '17 at 08:45

1 Answers1

1

That guide to multiple requests in Rcurl looks pretty simple, in fact I'd say it looks simpler to me than the solution to the Python question you've linked. Better yet, the work is already done for you. Most of that guide is going into detail about the advantages of concurrent requests; the method itself is deceptively simple, and is provided for you pre-cooked right at the top of the page.

You can literally cut and paste the code shown at the top of the post into an R script (include library(RCurl) above it), run that code to source the function, then call the function with a single line.

I won't paste the function code here, since you should get that from its author, but once you've sourced that function, their example usage is:

uris = c("http://www.omegahat.org/index.html", "http://www.omegahat.org/RecentActivities.html")

z <- getURIs(uris)

I just did the above on my own computer, and it works perfectly. I'd be surprised if you can find a simpler solution than that.

rosscova
  • 5,430
  • 1
  • 22
  • 35
  • Thanks, rosscova, you're right, on closer inspection, it's quite straightforward. The thing was that I had a loop using the `Rfacebook` package, and I wasn't dealing with the API directly. But you answered the question, thanks. – RobertMyles Jan 20 '17 at 21:50