Retrieving data in R from Import.io API using Rcurl?

Question

Im trying to retrieve data from an import.io 'connector' API.

Basically, i've trained the extractor to the structure of a given website, and I want to import the data from within R using this approach: 1) Retrieve the Json results from the API 2) Save each query-result into a given dataframe

My plan was to use RCurl for querying the API-link: https://api.import.io/store/connector/9128b4e0-9ae2-4232-b202-c1e8766ed01f/_query?input=webpage/url:[ENCODED URL]&&_apikey=[API-KEY]

require(Rcurl)
Raw.Data <- curl::curl(url = "https://api.import.io/store/connector/9128b4e0-9ae2-4232-b202-c1e8766ed01f/_query?input=webpage/url:[ENCODED URL]&&_apikey=[API-KEY]")

And after that, use Rjson for reading the retrieved data into a dataframe:

require(rjson)
FromJson_To_DataFrame <-(Raw.data)

There is something missing because I am getting errors, but i cannot figure out what it is, and if it is possible at all like this. Hints would be greatly aprecaiated!

Cheers from DK

What are the errors that you are getting? Also you have two ampersands in your query string `input=webpage/url:[ENCODED URL]&&_apikey=[API-KEY]` which might be causing an issue. — Bam4d, Feb 08 '16 at 16:59
@Bam4d thanks for reply! Youre right, the error was due to the two &&.. Now my next problem is to store the data in a dataframe which seems like its not so straight forward as just: **require(rjson) FromJson_To_DataFrame <-(Raw.data)** — MikeR, Feb 09 '16 at 21:21
I don't know much about R sorry, but I did find this which might be useful: http://stackoverflow.com/questions/16947643/getting-imported-json-data-into-a-data-frame-in-r The data format here is the same as that which is used in import.io datasets, so it will be a good start I hope! — Bam4d, Feb 10 '16 at 15:05

score 0 · Accepted Answer · edited Aug 15 '16 at 14:33

I found an answer to this problem, which is fairly straightforward. The json object retrieved from the import.io API can be changed in to columns in a DF by accesing it via $:

library(httr)   
output <- get(https://api.import.io/store/connector/9128b4e0-9ae2-4232-b202-c1e8766ed01f/_query?input=webpage/url:[ENCODED URL]&&_apikey=[API-KEY])

result <- content(output)

vector1 <- result$results$variable1
vector2 <- result$results$variable2

and then you can cbind them to a dataframe if you need to.

Retrieving data in R from Import.io API using Rcurl?

1 Answers1