Using Python to extract dictionary keys within a list

Question

I received a list when inputting the following URL - http://api.twitter.com/1/trends/44418.json

The list contains multiple dictionaries, and I'm a bit confused with the list structure. I'm trying to obtain the values associated with the 'name' key.

For example:

"name":"#throwagrenade" "name":"Rebecca Black" "name":"#questionsihate"

I can write the code myself, I'm just trying to conceptually understand how to access dictionaries (and their key/value pairs) within a list.

Just a note, twitter uses a API v1.1 now that requires authentication. You can read more about accessing the trends [here](https://dev.twitter.com/docs/api/1.1#110). — philshem, Aug 14 '13 at 08:40

score 22 · Accepted Answer · edited Nov 24 '11 at 10:09

The first thing I would do when working with a big lump of json, is try to get it into a more readable format. This online json formatting tool should do the job.

Here's some code that will get all the trend names:

import urllib2
import json

url = 'http://api.twitter.com/1/trends/44418.json'

# download the json string
json_string = urllib2.urlopen(url).read()

# de-serialize the string so that we can work with it
the_data = json.loads(json_string)

# get the list of trends
trends = the_data[0]['trends']

# print the name of each trend
for trend in trends:
    print trend['name']

Or you can do it all in one line:

names = [trend['name'] for trend in the_data[0]['trends']]

for name in names:
    print name

Both will result in:

#throwagrenade
Rebecca Black
Eric Abidal
#questionsihate
#juniordoctors
Smiley Culture
Lily Allen
Wes Brown
Pandev
Ray Wilkins

Relevant reading:

Python docs on json (although you should only really need json.loads())

Dive Into Python's sections on lists and dictionaries.

thanks! great response, appreciate the extra reading material as well. — Alex Karpowitsch, Mar 15 '11 at 23:08
I'd also recommend the new [Dive Into Python 3](https://diveintopython3.ep.io/) - dictionaries and lists are mentioned in the section [Chapter 2. Native Datatypes](https://diveintopython3.ep.io/native-datatypes.html). — joweiser, Feb 10 '12 at 20:36

score 8 · Answer 2 · answered Mar 15 '11 at 22:22

Well, for a start, that link gives you JSON, so you'll need to deserialize it with the json library:

data = json.loads(response_data)

Now you simply have a list of dictionaries. You can easily iterate through the list with a for loop. On each iteration, you have a normal dictionary, from which you can get the value corresponding to the name key with the usual dictionary syntax.

You can do the whole thing at once with a simple list comprehension:

names = [item['name'] for item in data]

The list of dictionaries containing the names are not in the root of the data, they are in `[0]['trends']` — Acorn, Mar 15 '11 at 23:53

score 4 · Answer 3 · answered Mar 15 '11 at 22:36

import urllib2
import json

url = 'http://api.twitter.com/1/trends/44418.json'
data = urllib2.urlopen(url).read()
j = json.loads(data)

names = [d['name'] for d in j[0]['trends']]

results in

names = [u'#throwagrenade', u'Rebecca Black', u'#questionsihate',
    u'#thingsthatdontgotogether', u'Eric Abidal', u'Smiley Culture',
    u'Ray Wilkins', u'Wes Brown', u'Twenty Twelve', u'Marseille']

score 1 · Answer 4 · answered Mar 15 '11 at 22:22

That's a JSON file, so you need to use a JSON parser to read it. There is a parser in Python 2.7 - just import json. With that structure, you can then manipulate it from Python.

If you really don't care where in the structure the name keys are, you could either recurse through the tree looking for them (if key == "name"), or perhaps use a regular expression.

The regular expression will be much pain, though, because of the need to include escaped characters in a match.

The idea of using a regular expression gives me a headache just thinking about it! — Acorn, Mar 15 '11 at 23:54

Using Python to extract dictionary keys within a list

4 Answers4

Linked