How to limit number of string values in API endpoint using Python?

Question

Here, in the code. I can only use 5 values maximum(API limitation) at a time in the below API.

import requests
import json
import datetime
from datetime import date, datetime
import itertools
from sentimeterdb import dbconnect

from dotenv import load_dotenv   #for python-dotenv method
load_dotenv()                    #for python-dotenv method

import os

#Connection string pointing to the PROD db.
dbclient = dbconnect.process()
db = dbclient.abc
co_open = db.open_ab

myquery = {"bugs" : {"$ne" : {}, "$exists": True}}
mydoc = co_open.find(myquery)

lst = []
for x in mydoc:
    y = x.get('bugs')
    id_list = lst.append([*y.values()])
    cdets = list(itertools.chain(*lst))

cdets_split = cdets
string_cdets = ','.join(cdets_split[:5])

Access_URL = 'https://generating-token-from--here/oauth2/default/v1/token'

client_id = os.environ.get('CLIENT_ID')
client_secret = os.environ.get('CLIENT_SECRET')
grant_type = os.environ.get('GRANT_TYPE')
BASE_URL = f'https://google.com/{string_cdets}'

response = requests.post(Access_URL,
                     data={'grant_type': grant_type, 'client_id': client_id, 'client_secret': client_secret, 'content-type': 'application/x-www-form-urlencoded'})
json_response = response.json()

tokenvalue = (json_response['access_token'])

headers = {'Authorization': 'Bearer ' +
       tokenvalue, 'Content-Type': 'application/json'}

auth_response = requests.get(BASE_URL,  headers=headers)
auth_response_json = auth_response.json()

severity_list = []
for severity_5 in auth_response_json['bugs']:
    severity_list.append([severity_5['severity']])

print ("Severity ==> ", severity_list)

Severity and Status should be given for all the CDETs. Currently, It is only giving for 5 items. how can I get the response of all the CDETs?

score 0 · Answer 1 · answered May 25 '23 at 14:50

0

I am not sure if I get the question, but using a generator does not affect the complexity at all. A for loop is perfectly fine.

A generator would be useful, if the values ('ABC', 'DEF' etc.) were generated on-the-fly. In that case, you could use a for loop to cycle over the generator.

answered May 25 '23 at 14:50

PetrPy

1
1

Even If I use for loop, can you demonstrate how it is possible using the date I provided ? – adiSF May 25 '23 at 16:43

JRiggles · Answer 2 · 2023-05-25T18:34:31.567

There is a recipe shown in the itertools documentation for a function called batched which can be used to yield "chunks" of an arbitrary size from an iterable

from itertools import islice 


def batched(iterable, n):
    "Batch data into tuples of length n. The last batch may be shorter."
    # batched('ABCDEFG', 3) --> ABC DEF G
    if n < 1:
        raise ValueError('n must be at least one')
    it = iter(iterable)
    while batch := tuple(islice(it, n)):
        yield batch

The answer given here might also be useful.

At any rate, as you've already guessed, you'll end up passing these chunks into a for loop to make your API call(s).

for group in batched(values, 5):
    ...  # insert API call here

I'm not sure of a way you can do this without multiple API calls. Taking the approach of chunking the list like this inherently means you'll have to make the call until your list is exhausted.

EDIT: following updates from OP

If what's changing is the value of BASE_URL, you would simply loop over your "batches" from cdets, updating BASE_URL as needed and passing that to requests.get(BASE_URL, headers=headers)

auth_responses = []  # init an empty list to store responses


for batch in batched(cdets, 5):
    string_cdets = ','.join(batch)
    BASE_URL = f'https://google.com/{string_cdets}'
    # append responses to the list
    auth_responses.extend(requests.get(BASE_URL, headers=headers))

Caveat - there's a bit of guesswork done here since we have no idea what these CDETs are...

Also FWIW, BASE_URL should really be styled as base_url since it's a variable whose value changes - all caps is typically used to imply a CONSTANT value, which BASE_URL definitely isn't

Can you demonstrate the code with API call as well, I am little confuse with the appropriate data type which API requires. Ex;- api = https://www.google.com/ABC,DEF,GHI,JKL,MNO — adiSF, May 25 '23 at 17:14
You'll have to edit your question to include the code you use for the API calls. As it stands, all `api = https://www.google.com/{val}` is doing is formatting a string. And without knowing what API you're using, we can't help much. — JRiggles, May 25 '23 at 17:16
Just edited the question for better understanding. Let me know If anything is not clear. — adiSF, May 25 '23 at 18:22
See update - I hope that helps, as I can't really test it on my end — JRiggles, May 25 '23 at 18:34

How to limit number of string values in API endpoint using Python?

2 Answers2