1

I'm currently working with JSON API data which updated daily by my data team. I only need to extract certain information from the JSON file using pandas. So I managed to transform the data by pulling 1 sample JSON file from the API provided. However, I also need to update the transformed data in pandas with other JSON files from the API and I'm stuck here. FYI the JSON files from the API contain in folders name by according to date pulled

The following is the code I used to pull 1 sample JSON data for the transformation process. 

#pull 1 sample data from API
df = requests.get('http://phpweb.123.co/atz-pull/output/2019-07-30/abc123.json').json()

So I managed to transform the data into the form i required. However I need to update the transformed data with other json files from different date folders.

Let say the output folders from the api also consist of the following (note that the json file name is unique ;

/output/2019-07-30/abc456.json
/output/2019-07-30/def678.json 
/output/2019-07-29/wef678.json

Appreciate helps

m36a
  • 35
  • 5
  • Check this out https://stackoverflow.com/questions/41168558/python-how-to-convert-json-file-to-dataframe/41168691 – Anshuman Kumar Jul 31 '19 at 05:13
  • 1
    thanks , i managed to do that already .. however what i mean is after all the transformation , i need to update the transformed data with other json files from the API . – m36a Jul 31 '19 at 05:24
  • The process in my mind somehow like this; read json data on 29/7/2019 --- process to transform to pandas dataframe ---- read json data on 30/7/2019 --- process to transform to pandas dataframe ---- concat with transfrom pandas df on 29/7/2019 .... and so on – m36a Jul 31 '19 at 05:30
  • 1
    https://stackoverflow.com/questions/56866697/python-read-json-files-from-all-sub-directories this link actually answer my question – m36a Jul 31 '19 at 10:26

0 Answers0