How to parse json to get all values of a specific key within an array?

Question

I'm having trouble trying to get a list of values from a specific key inside an json array using python. Using the JSON example below, I am trying to create a list which consists only the values of the name key. Original JSON:

[
    {
        "id": 1,
        "name": "Bulbasaur",
        "type": [
            "grass",
            "poison"
        ]
    },
    {
        "id": 2,
        "name": "Ivysaur",
        "type": [
            "grass",
            "poison"
        ]
    }
]

Expected:

["Bulbasaur", "Ivysaur"]

Below is the code of my approach:

import json
try:
    with open("./simple.json", 'r') as f:
        contents = json.load(f)
except Exception as e:
    print(e)

print(contents[:]["name"])

I'm trying to go to an approach where i don't need to loop every single index and append them, something like the code above. Is this approach possible using python' json library?

It's not possible in vanilla Python because lists don't do that. — Mad Physicist, May 16 '19 at 07:49
The behaviour would be undefined if say one of the elements wasn't a dictionary or was missing that particular key. Python doesn't do well with ambiguous behaviors. — Mad Physicist, May 16 '19 at 07:50
@MadPhysicist when you say `vanilla` Python doesn't do that, is there a version of `Python` which does that? — Devesh Kumar Singh, May 16 '19 at 08:14
@Devesh. Not a version of Python as much as an external library. I'm not sure of one specifically, but I can easily write you a dict subclass that returns an iterable that allows the further index assignment like this one. It's reminiscent of numpy or pandas, though obviously not in the details. — Mad Physicist, May 16 '19 at 13:14
@Devesh If you're interested, I could add an answer. +1 to yours BTW. — Mad Physicist, May 16 '19 at 13:17
That's fine, I think I will try to attempt it myself @MadPhysicist :) — Devesh Kumar Singh, May 16 '19 at 13:18

score 17 · Accepted Answer · answered May 16 '19 at 07:49

17

You cannot do contents[:]["name"] since contents is a list is a dictionary with integer indexes, and you cannot access an element from it using a string name.

To fix that, you would want to iterate over the list and get the value for key name for each item

import json
contents = []

try:
    with open("./simple.json", 'r') as f:
        contents = json.load(f)
except Exception as e:
    print(e)


li = [item.get('name') for item in contents]
print(li)

The output will be

['Bulbasaur', 'Ivysaur']

answered May 16 '19 at 07:49

Devesh Kumar Singh

20,259
5
21
40

1

I couldn't resist posting an answer. – Mad Physicist May 16 '19 at 21:46
Thanks @MadPhysicist you answer almost cost me an accepted answer lol :) But thanks for the teachable moment in form of your answer :) – Devesh Kumar Singh May 18 '19 at 05:35
I'm glad you got it back :) – Mad Physicist May 18 '19 at 13:21

Mad Physicist · Answer 2 · 2019-05-17T00:53:08.797

This is not a real answer to the question. The real answer is to use a list comprehension. However, you can make a class that allows you to use specifically the syntax you tried in the question. The general idea is to subclass list so that a slice like [:] returns a special view (another class) into the list. This special view will then allow retrieval and assignment from all the dictionaries simultaneously.

class DictView:
    """
    A special class for getting and setting multiple dictionaries
    simultaneously. This class is not meant to be instantiated
    in its own, but rather in response to a slice operation on UniformDictList.
    """
    def __init__(parent, slice):
        self.parent = parent
        self.range = range(*slice.indices(len(parent)))

    def keys(self):
        """
        Retreives a set of all the keys that are shared across all
        indexed dictionaries. This method makes `DictView` appear as
        a genuine mapping type to `dict`.
        """
        key_set = set()
        for k in self.range:
            key_set &= self.parent.keys()
        return key_set

    def __getitem__(self, key):
        """
        Retreives a list of values corresponding to all the indexed
        values for `key` in the parent. Any missing key will raise
        a `KeyError`.
        """
        return [self.parent[k][key] for k in self.range]

    def get(self, key, default=None):
        """
        Retreives a list of values corresponding to all the indexed
        values for `key` in the parent. Any missing key will return
        `default`.
        """
        return [self.parent[k].get(key, default) for k in self.range]

    def __setitem__(self, key, value):
        """
        Set all the values in the indexed dictionaries for `key` to `value`.
        """
        for k in self.range:
            self.parent[k][key] = value

    def update(self, *args, **kwargs):
        """
        Update all the indexed dictionaries in the parent with the specified
        values. Arguments are the same as to `dict.update`.
        """
        for k in self.range:
             self.parent[k].update(*args, **kwargs)


class UniformDictList(list):
    def __getitem__(self, key):
        if isinstance(key, slice):
            return DictView(self, key)
        return super().__getitem__(key)

Your original code would now work out of the box with just one additional wrap in UniformDictList:

import json
try:
    with open("./simple.json", 'r') as f:
        contents = UniformDictList(json.load(f))
except Exception as e:
    print(e)

print(contents[:]["name"])

This is slightly too advanced for me @MadPhysicist could you share some tutorial or docs where I can learn more about these things — Devesh Kumar Singh, May 17 '19 at 02:59
@Devesh. This is just cleverly overriding the `[]` operator. Here are a couple of references: https://docs.python.org/3/glossary.html#term-slice, https://docs.python.org/3/library/functions.html#slice, https://docs.python.org/3/reference/datamodel.html — Mad Physicist, May 17 '19 at 13:19
@Devesh. The `indices` method of `slice` appears to be undocumented on the Python side. Not sure why. I'm there data model, focus particularly on `__getitem__` and `__setitem__`. They override the operators that OP wants to use. — Mad Physicist, May 17 '19 at 13:20
@MadPhysicist Thanks so much for your answer! Is it safe to say that the performance doesn't affect too much with this code? — Radhian Amri, May 17 '19 at 15:38
Hi @RadhianAmri , as the poster already specified in his answer, you really do not need to create a new datatype to cater a specific problem, and the ideal solution is to create a list comprehension like what was done in my answer. The poster just posted this on my request for me to understand how could it be done in a specific manner, but this is not a suggestion of doing it this way :) — Devesh Kumar Singh, May 17 '19 at 15:50
@Radhian. I have to vehemently agree with Devesh here. This is just an unnecessarily convoluted toy I wrote to show that it *could* be done. It will likely not impact your performance much, but whatever impact it does have won't be positive. Don't use this unless you really need to (you don't). Please select Devesh's answer again. — Mad Physicist, May 17 '19 at 16:34
Hi @RadhianAmri as Mad Physicist said, this will do you more harm then good to solve your simplified problem. Please mark my answer as accepted again :) — Devesh Kumar Singh, May 17 '19 at 17:46

score 1 · Answer 3 · answered May 16 '19 at 07:50

1

Try this with list comprehensions:

print([d["name"] for d in contents])

answered May 16 '19 at 07:50

olinox14

6,177
2
22
39

How to parse json to get all values of a specific key within an array?

3 Answers3

Linked

Related