0

I would like to fetch data from a Reddit page but I'm having a problem converting into JSON with the RJSONIO package. It works for others Reddit pages but this one is causing me headache because it return invalid JSON input. 

library(RCurl)
library(RJSONIO)

json <- getURL("https://www.reddit.com/r/burstcoin/new.json?sort=rising?limit=100")

list <- RJSONIO::fromJSON(json)
Error in fromJSON(content, handler, default.size, depth, allowComments,  : 
  invalid JSON input

> Encoding(json)
[1] "UTF-8"

I've try to remove non-ASCII characters with iconv but it doesn't give the expected result and I still have the same error.

json <- iconv(json, "UTF-8", "ASCII", sub="")

> Encoding(json)
[1] "unknown"

I also use the stringi package which provides a function for general unicode conversion but still the same error.

library(stringi)
json <- stringi::stri_trans_general(json, "latin-ascii")

How can I parse this JSON file ?

Similar question : Removing non-ASCII characters from data files

Florent
  • 1,791
  • 22
  • 40
  • list <- fromJSON(toJSON(json)) – Cybernetic Jan 16 '18 at 06:53
  • Using jsonlite instead of RJSONIO. Returns a character vector. – Cybernetic Jan 16 '18 at 06:59
  • I have an error with `jsonlite` as well : `lexical error: invalid character inside string.` – Florent Jan 16 '18 at 08:52
  • Did you wrap your fromJSON around toJSON? I used your link and the lexical error disappears when I use list <- fromJSON(toJSON(json) – Cybernetic Jan 16 '18 at 15:04
  • Correct but I need to run `json <- fromJSON(toJSON(json))` to remove invalid character and then `list <- RJSONIO::fromJSON(json)` to get a list. Great, thanks ! Could you explain how it works ? – Florent Jan 16 '18 at 21:44
  • 1
    Data structures in R do not perfectly map to their JSON counterparts, and leave some ambiguity for edge cases. The jsonlite library was developed to offer a more concrete mapping, not available in other R json libraries. My assumption is jsonlite's more refined implementation of parsing/encoding preps your json so that your final mapping to a list structure is made possible. https://arxiv.org/pdf/1403.2805.pdf. – Cybernetic Jan 16 '18 at 22:43

1 Answers1

0

The answer has been given by user @Cybernetic in the comment section and works perfectly : 

json <- getURL("https://www.reddit.com/r/burstcoin/new.json?sort=rising?limit=100")
json <- fromJSON(toJSON(json))
list <- RJSONIO::fromJSON(json)
Florent
  • 1,791
  • 22
  • 40