Extract value from json response python?

Question

I am making a request to an endpoint and getting a json response in the following format.

r = requests.get(url)
print(r.json())

{"context":"product=fhvh, price=25,product=vfjvnf,price=37,product=fvfv,price=3.....}

I want to loop over them and get the count of the products that have a price point of more than 22.

How can i get that? I tried to loop over the json response initially but i don't believe that will work due to the format of the output. Is there a better solution? How can i convert the response to maybe a dict or tuple first? Can someone help me. Thank you.

Do you know the structure of the response? Can you provide the skeleton? — Prav, May 17 '22 at 23:39
@PraveenPremaratne Yes, i have. This is the skeleton :- ```{"context":"product=fhvh, price=25,product=vfjvnf,price=37,product=fvfv,price=3.....}``` — Somethingwhatever, May 17 '22 at 23:40
Do you know if it always comes in pairs? `product=fhvh, price=25` — Prav, May 17 '22 at 23:44
It looks like this question is more about string processing rather than JSON. — PIG208, May 17 '22 at 23:44
So your response is coming in as an unordered ordered list within a string with no validation (e.g. product=fhvh, etc.)? Can you just post the actual JSON? — psychopg2, May 17 '22 at 23:47
@psychopg2 Unfortunately this the actual data that i am getting. — Somethingwhatever, May 17 '22 at 23:48
I've a solution but its not an ideal or one I'd personally use — Prav, May 17 '22 at 23:48

Prav · Accepted Answer · 2022-05-18T00:09:31.423

2

This is the worst possible solution, it relies on the response always has a product and a price and can be very unreliable.

r = requests.get(url)
data = r.json()

context=data.get('context')
elements = context.split(',')
chunks=[elements[i:i + 2] for i in range(0, len(elements), 2)]
print(chunks)

[['product=fhvh', ' price=25'], ['product=vfjvnf', 'price=37'], ['product=fvfv', 'price=3']]

I'd look into data processing libraries used by Big Data like Pandas that might have a more reliable way to doing it.

Update

r = requests.get(url)
data = r.json()

context=data.get('context')
elements = context.split(',')
chunks=[elements[i:i + 2] for i in range(0, len(elements), 2)]
print(chunks)

product_list = []
for item in chunks:
     obj1=item[0].split('=')
     obj2=item[1].split('=')
     product = { obj1[0].strip(): obj1[1].strip(), obj2[0].strip(): float(obj2[1].strip())}
     product_list.append(product)

print(product_list)

for product in product_list:
     if product.get('price') > 22.0:
             print(product)

edited May 18 '22 at 00:09

answered May 17 '22 at 23:50

Prav

2,785
1
21
30

Hmmm, so you converted it into smaller lists. How can i get the product whose price is more than 22? – Somethingwhatever May 17 '22 at 23:51
It does always have a price and product. – Somethingwhatever May 17 '22 at 23:53
It's an array of arrays... so you can just iterate and build another array that has elements that are over a certain price. – Prav May 17 '22 at 23:53
1

Yes, as long as the response provides it in that sort of order. So if the response dos this `product=fhvh, product=vfjvnf, price=25, price=37` you're screwed. – Prav May 17 '22 at 23:54
This is why its very unreliable, you're putting faith in the API to always be consistent – Prav May 17 '22 at 23:55
1

Well i will still accept this as a potential answer and will look into pandas. – Somethingwhatever May 17 '22 at 23:57
@Somethingwhatever check the update, that should work. – Prav May 18 '22 at 00:09

Woody1193 · Answer 2 · 2022-05-18T00:17:53.957

Your issue is that the payload you have isn't JSON but some alternative format. My advice would be to convert each entry in context into an object:

class Product:
    def __init__(self, name, price):
        self.name = name
        self.price = float(price)

def parse_data(resp):

    # First, read in the JSON
    raw_data = r.json()
 
    # Next, split the context field in the JSON into a list-of-tuples
    # where each item represents a key and value.
    parsed_data = [tuple(item.strip().split('=')) for item in raw_data["context"].split(',')]
   
    # Finally, convert each product to our type and return the list of them
    # This assumes that the payload contains a product followed by a price
    return [Product([i][1], y[i + 1][1]) for i in range(0, len(y), 2)]

r = requests.get(url)
filtered = [product for product in parse_data(r) if product.price > 22]

This code obeys separation of concerns and produces the output you want (all products with a price greater than 22). That being said, this relies heavily on assumptions about what is returned from the API you're requesting. You'll want to add data checks to ensure that you're not getting bad data.

This looks much better than mine, but I'd parse the price as a `float`. It may have decimals. — Prav, May 18 '22 at 00:12
@PraveenPremaratne I think that's a valid point. Actually, I don't think I did any parsing of the value at all. — Woody1193, May 18 '22 at 00:17

Extract value from json response python?

2 Answers2

Update

Linked