2

I'm trying to manipulate data from large .json files, but when the file gets too big, instead of giving me all the data, read_json only gives the first and last five lines. Is there some way to force the read_json to provide all rows form 0 to 485?

code

import pandas as pd
hist = pd.read_json('file_address.json')
print(hist)

results:

                                                 candles symbol  empty  
  0    {'open': 115.12, 'high': 115.12, 'low': 115.12...    CCF  False  
  1    {'open': 115.85, 'high': 115.85, 'low': 115.85...    CCF  False  
  2    {'open': 115.3, 'high': 115.94, 'low': 115.3, ...    CCF  False  
  3    {'open': 114.91, 'high': 114.91, 'low': 114.91...    CCF  False  
  4    {'open': 115.33, 'high': 115.33, 'low': 115.33...    CCF  False  
  ..                                                 ...    ...    ...  
  481  {'open': 102.79, 'high': 102.79, 'low': 102.04...    CCF  False  
  482  {'open': 101.58, 'high': 102.04, 'low': 101.58...    CCF  False  
  483  {'open': 102.02, 'high': 102.475, 'low': 102.0...    CCF  False  
  484  {'open': 102.57, 'high': 102.57, 'low': 102.57...    CCF  False  
  485  {'open': 102.58, 'high': 102.58, 'low': 102.58...    CCF  False  

[486 rows x 3 columns]
Jon
  • 45
  • 5
  • does this solve the issue: https://stackoverflow.com/questions/11707586/how-do-i-expand-the-output-display-to-see-more-columns-of-a-pandas-dataframe? – Krishna Chaurasia Jan 30 '21 at 07:21

1 Answers1

2

read_json is correctly reading the whole JSON. However, Pandas has an option called display.max_rows that limits the number of lines that will be displayed when printing a dataframe.

You can remove this limit with the set_option method.

Here is the code you can use, if you want to display all the lines of the dataframe:

import pandas as pd
pandas.set_option('display.max_rows', None)
hist = pd.read_json('file_address.json')
print(hist)
Marc Dillar
  • 471
  • 2
  • 9