0

I need to load a large JSON file in a pandas dataframe.

My question is how to read/ open this file in pandas ?

With the terminal, I can see the head of the file (each row/record is separed by\n)

{'asin': '0001048791', 'salesRank': {'Books': 6334800}, 'imUrl': 
'http://ecx.images-amazon.com/images/I/51MKP0T4DBL.jpg', 'categories':     
[['Books']], 'title': 'The Crucible: Performed by Stuart Pankin, Jerome 
Dempsey & Cast'}
{'asin': '0000143561', 'categories': [['Movies & TV', 'Movies']],
'description': '3Pack DVD set - Italian Classics, Parties and Holidays.', 
'title': 'Everyday Italian (with Giada de Laurentiis), Volume 1 (3 Pack): 
Italian Classics, Parties, Holidays', 'price': 12.99, 'salesRank': {'Movies 
& TV': 376041}, 'imUrl': 'http://g-ecx.images-amazon.com/images/G/01/x- 
site/icons/no-img-sm._CB192198896_.gif', 'related': {'also_viewed': 
['B0036FO6SI', 'B000KL8ODE', '000014357X', 'B0037718RC', 'B002I5GNVU', 
'B000RBU4BM'], 'buy_after_viewing': ['B0036FO6SI', 'B000KL8ODE', 
'000014357X', 'B0037718RC']}}

Using read_json from pandas give me an error (ValueError: Expected object or value), including when I use different values for the orient parameter.

import pandas as pd
df = pd.read_json('metadata.json', lines=True)
SciPy
  • 5,412
  • 4
  • 18
  • 18
  • Will you be able to provide the file http link? – Karn Kumar Dec 30 '18 at 17:54
  • 1
    You have a typo in the second line of code (`'metadata.json`). This is not the reason of your error though. Moreover, the problem you have reported is not related to the size of the file. Please, take a look at this: https://stackoverflow.com/questions/28373282/how-to-read-a-json-dictionary-type-file-with-pandas – floatingpurr Dec 30 '18 at 18:12

0 Answers0