0

I am calling an API with while loop, getting json, and converting it to df. How can I stick dataframes - see the part objects- resulted from every iteration? 

#importing libraries
import requests
import pandas as pd
import numpy as np
import requests
import json

#getting the max available num of pages
response = requests.get('url?page%5Bnumber%5D=10000&page%5Bsize%5D=30', 
                        headers={'Authorization': 'Token token=x' , 'X-Api-Version': 20161108'}).json()
page_num = response['meta']['page-count']

#initiating the count
i = 1
fmt = 'url?page%5Bsize%5D=30&page%5Bnumber%5D={page_number}'
while i <= page_num:
    values = { 'page_number': i}
    formatted = fmt.format(**values)
    response = requests.get(formatted, 
                        headers={'Authorization': 'Token token=x', 'X-Api-Version': '20161108'}).json()
    objects = json_normalize(response['data'])
    i += 1
eponkratova
  • 467
  • 7
  • 20
  • "How can I stick dataframes - see the part objects- resulted from every iteration?" I'm not sure what you're asking here. Perhaps in answering your own question you can strip away all the complexity of JSON and HTTP requests. Because what's the result of all that? Some `dict`. Okay, so you're doing something with `dict`s in a loop. It's not clear to me WHAT you want to be doing but the point is your question doesn't seem to have anything to do with anything but that. – Iguananaut Dec 26 '19 at 11:16
  • I guess maybe you want to convert "the part objects" to a DataFrame or something but I can't possibly tell you how to do that without knowing the format of these objects or what end result you want. – Iguananaut Dec 26 '19 at 11:18
  • @Iguananaut, thank you for the response. Each page retrieved is stored in objects. I already flattened the json obect into a dataframe - see the part objects = json.... What I need is to stick each iteration (=a dataframe from each iteration) on top of each other. – eponkratova Dec 26 '19 at 11:27
  • I see. That wasn't clear because you didn't say what `json_normalize` is. I hope you can see that, in that case, that you could have then simplified your question to how to concatenate multiple DataFrames (regardless of how they were created). All the JSON stuff as far as I can tell is not relevant to your question. I say this because it will help you in future when seeking answers to problems to be able to think more clearly about how to logically isolate individual parts of your problem from each other. – Iguananaut Dec 26 '19 at 11:34
  • Beautiful, @Iguananaut, it worked out. I have not seen that question. – eponkratova Dec 26 '19 at 11:41

0 Answers0