0

I have a JSON dataset like the following:

{"reviewerID": "A23I5I4K77N533", "asin": "1412719410", "reviewerName": "J. Morton \"Aussie mama\"", "helpful": [4, 4], "reviewText": "This has to be one of the most frustrating toys. Ours is the version with Winnie the Pooh books, but the hardware is the same. In theory it is a good idea, but it does not even come close to living up to that. Ninety percent of the time it will not recognize that there is a book inserted and will keep repeating, \"Please insert a book.\" When it does start to read you only have to jiggle the book ever so slightly for it to ask for a book again.The \"cover\" of the system comes apart from the base very easily.I would not recommend this toy at all.", "overall": 1.0, "summary": "Worst Toy Ever.", "unixReviewTime": 1343606400, "reviewTime": "07 30, 2012"} {"reviewerID": "A18PMN2VYJ73Z6", "asin": "1412719410", "reviewerName": "lalisaww", "helpful": [0, 0], "reviewText": "It does nothing but saying \"bye bye\" after a long silence.Quite odd, isn't it?I returned it to amazon after trying for a few times.Awful toy.", "overall": 1.0, "summary": "It doesn't work at all", "unixReviewTime": 1379030400, "reviewTime": "09 13, 2013"}

Its 2.5 million+ data, size 1.7gb+. So, editing or manipulating the data manually is very difficult. I tried loading it in VS code which gives me this error:

JSONDecodeError: Unterminated string starting at: line 1 column 116

My code for loading the data was:

1.

reviews= []
for line in open('ReviewSample.json', 'r'):
    reviews.append(json.loads(line))

2.

data = [json.loads(line) for line in open('ReviewSample.json', 'r')]

both worked fine in Jupyter(Anaconda) but gives error in VS Code(in Jupyter servers as well),
Any effective solution to make it work in VS Code? TIA.

Saqueeb
  • 1
  • 3
  • You're trying to read line by line - what if a line doesn't contain a complete JSON object? See the duplicate and the documentation for the correct way to use the json module. – Thierry Lathuille May 26 '20 at 07:56
  • @ThierryLathuille I tried other solutions like : with open('strings.json') as f: d = json.load(f) print(d) but gives error: JSONDecodeError: Extra data: line 2 column 1 (char 330) – Saqueeb May 26 '20 at 08:05
  • Why it works in Jupyter Anaconda but not in VS code with Jupyter server? It doesn't make any sense. – Saqueeb May 26 '20 at 08:08
  • Are you sure that you are using the same JSON file (in the same directory) in both cases? Also, could you provide a small sample file that reproduces the problem? The one you pasted here looks like the output of print, not like a real part of the file that should contain double backslashes for the escaped quotes. – Thierry Lathuille May 26 '20 at 08:22
  • I am sure, both are for the same file, same directory, working for Jupyter and not working for VS Code, the error I am getting is mentioned. Which command worked for Jupyter also mentioned and exact same code when I implement it in VS Code shows error. Nah, it's fine, I just pasted it in the comment from where you said its duplicate. I tried every possible answer in this site that can be the solution but it's not working. What should I provide you for better understanding? – Saqueeb May 26 '20 at 09:49

0 Answers0