0

I have a response that I need to page through but it doesn't have the total page number available. How can I go about paging through the response? For instance lets say I have this response:

response = requests.get(url='test.com', headers=headers, page=1)

and I want to do something like this where I get each page of the response's results into the items list, but as I mentioned I don't know what my total_pages are, so how would get each pages response and stop when the page is empty?

items = []
for page_number in range(1, total_pages + 1): 
     items = items + requests.get(url='test.com', headers=headers, page=page_number)
orlp
  • 112,504
  • 36
  • 218
  • 315
KristiLuna
  • 1,601
  • 2
  • 18
  • 52
  • 1
    Does this answer your question? [Repeat-until or equivalent loop in Python](https://stackoverflow.com/questions/16758807/repeat-until-or-equivalent-loop-in-python) – mkrieger1 Feb 17 '21 at 20:25

1 Answers1

1

Use a while loop.

items = []
page_nr = 1
while True:
    page = requests.get(url='test.com', headers=headers, page=page_nr)
    is_empty = # ... your code here to check if the page is empty.
    if is_empty:
        break
    # ... add stuff from page to items.
    page_nr += 1
orlp
  • 112,504
  • 36
  • 218
  • 315