0

I am using the tweetscores package of R to get 'tweets list from twitter. The tweets are stored in json format. While converting it to a data frame I get a lexical error ' Error: lexical error: inside a string, '\' occurs before a character which it may not.".

Any solution to the mentioned error.

A part of the json file text

":[{"text":["MUFC"],"indices":[[83],[88]]}],"symbols":[],"user_mentions":[],"urls":[]},"metadata":{"iso_language_code":["en"],"result_type":["recent"]},"source":["http://twitter.com/download/iphone\" rel=\"nofollow\">Twitter for iPhone</a>"],"in_reply_to_status_id":null,"in_reply_to_status_id_str":null,"in_reply_to_user_id":null,"in_reply_to_user_id_str":null,"in_reply_to_screen_name":null,"user":{"id":[7.32108114527322e+017],"id_str":["732108114527322112"],"name":["wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww(^o^)/"],"screen_name":["SukiSukinal"],"location":["+6222"],"description":["Alliansi osaosi ngevote kagak. katanya sih fans a.k.a + "],"url":null,"entities":{"description":{"urls":[]}},"protected":[false],"followers_count":[163],"friends_count":[107],"listed_count":[4],"created_at":["Mon May 16 07:19:11 +0000

SymbolixAU
  • 25,502
  • 4
  • 67
  • 139
Devarshi Mandal
  • 703
  • 8
  • 16
  • Welcome to Stack Overflow. You should provide a reproducible example of your data as described [here](https://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example) and show what you have tried so far. – tobiasegli_te Nov 19 '17 at 14:31
  • Hi, I am using the following code to convert the json file into a data frame tweet_file <- "tweetlist.json" con <- file(tweet_file,"r") input <- readLines(con, -3L) close(con) tweetdata <- ldply(lapply(input, function (x) t(unlist(fromJSON(x, flatten = TRUE))))) – Devarshi Mandal Nov 19 '17 at 14:34
  • iPhone<\/a>"],"in_reply_to_status_id":null,"in_reply_to_status_id_str":null,"in_reply_to_user_id":null,"in_reply_to_user_id_str":null,"in_reply_to_screen_name":null,"user":{"id":[7.32108114527322e+017],"id_str":["732108114527322112"],"name":["wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww\(^o^)/"],"screen_name":["SukiSukinal"],"location":["+6222"],"description":["Alliansi osaosi ngevote kagak. katanya sih fans a.k.a + – Devarshi Mandal Nov 19 '17 at 14:45
  • Don't use the comments section to add details to your question. Edit your question (by pressing 'edit') with the extra details and code you've tried. – SymbolixAU Nov 20 '17 at 00:33

2 Answers2

0

Json format does not allow backslashes so you need to escape them. replace any '\' character found with '\\'. Refer [here][1]

[1]: http://www.json.org/ for more info

vhoxha
  • 60
  • 2
  • 10
  • I agree 'backslashes' are not allowed, i tried replacing them using 'gsub' but it is not working – Devarshi Mandal Nov 19 '17 at 14:37
  • "Tried with gsub" is ambiguous. How? Remember that regex patterns sent to any rgex need to have any backslash "escaped". This replaces a single backslash with an "a": `gsub("\\\\", "a", "\\") [1] "a"` – IRTFM May 08 '18 at 20:02
0

You likely have an incomplete json string, which may be caused by the package or by an interrupted connection to Twitter's API. A complete json string returned from Twitter should look something like the following:

complete Twitter json document

which I got using rtweet's stream_tweets() function. With a complete string returned by Twitter's REST or stream API, you should be able to convert the data using basically any json parser (e.g., jsonlite::fromJSON()).

mkearney
  • 1,266
  • 10
  • 18