3

I could read this nested JSON format in columnar format into pandas.

JSON Scheme

JSON scheme format

enter image description here

Python script

    req = requests.get(REQUEST_API)
    returned_data = json.loads(req.text)
    # status
    print("status: {0}".format(returned_data["status"]))
    # api version
    print("version: {0}".format(returned_data["version"]))
    data_in_columnar_form = pd.DataFrame(returned_data["data"])
    data = data_in_columnar_form["data"]

UPDATE

I want to convert the following JSON scheme into the tabular format as the table, how to ?

inline

JSON Scheme

     "data":[  
        {  
           "year":"2009",
           "values":[  
              {  
                 "Actual":"(0.2)"
              },
              {  
                 "Upper End of Range":"-"
              },
              {  
                 "Upper End of Central Tendency":"-"
              },
              {  
                 "Lower End of Central Tendency":"-"
              },
              {  
                 "Lower End of Range":"-"
              }
           ]
        },
        {  
           "year":"2010",
           "values":[  
              {  
                 "Actual":"2.8"
              },
              {  
                 "Upper End of Range":"-"
              },
              {  
                 "Upper End of Central Tendency":"-"
              },
              {  
                 "Lower End of Central Tendency":"-"
              },
              {  
                 "Lower End of Range":"-"
              }
           ]
        },...
        ]
user3675188
  • 7,271
  • 11
  • 40
  • 76
  • Maybe this is what you are looking for? [Construct pandas DataFrame from items in nested dictionary](http://stackoverflow.com/q/13575090/222914) – Janne Karila Feb 04 '15 at 12:43

1 Answers1

8

Pandas has a JSON normalization function (as of 0.13), straight out of the docs:

In [205]: from pandas.io.json import json_normalize

In [206]: data = [{'state': 'Florida',
   .....:           'shortname': 'FL',
   .....:           'info': {
   .....:                'governor': 'Rick Scott'
   .....:           },
   .....:           'counties': [{'name': 'Dade', 'population': 12345},
   .....:                       {'name': 'Broward', 'population': 40000},
   .....:                       {'name': 'Palm Beach', 'population': 60000}]},
   .....:          {'state': 'Ohio',
   .....:           'shortname': 'OH',
   .....:           'info': {
   .....:                'governor': 'John Kasich'
   .....:           },
   .....:           'counties': [{'name': 'Summit', 'population': 1234},
   .....:                        {'name': 'Cuyahoga', 'population': 1337}]}]
   .....: 

In [207]: json_normalize(data, 'counties', ['state', 'shortname', ['info', 'governor']])
Out[207]: 
         name  population info.governor    state shortname
0        Dade       12345    Rick Scott  Florida        FL
1     Broward       40000    Rick Scott  Florida        FL
2  Palm Beach       60000    Rick Scott  Florida        FL
3      Summit        1234   John Kasich     Ohio        OH
4    Cuyahoga        1337   John Kasich     Ohio        OH
Andy Hayden
  • 359,921
  • 101
  • 625
  • 535
  • I always forget this exists and so I've reimplemented it a few times. :-( – DSM Feb 04 '15 at 17:50
  • @DSM me too... to be fair, it's "recent". :) – Andy Hayden Feb 04 '15 at 17:53
  • @AndyHayden thanks for your reply, could you give me some hint for the new question http://stackoverflow.com/questions/28335321/how-to-convert-nested-dictionary-into-a-2d-table – user3675188 Feb 05 '15 at 02:45
  • 1
    @Andy Hayden What if ```'counties'``` is a child of the ```'info'```? What would the meta be? I am still testing that to parse the ```'info':['counties']``` values. – Datacrawler Aug 01 '19 at 10:41