2

How can I get the number of hits for a Google query using Python?

I have a list containing the titles of songs from The Beatles:

    songs = ["Love Me Do", "P. S. I Love You", "Please Please Me"]

And I would like to retrieve the number of Google hits when I search Google for:

    The Beatles Love Me Do
    The Beatles P. S. I Love You
    The Beatles Please Please Me

And store these hits into another list. So in the end I'd end up with:

    google_hits == [42400000, 2740000, 28200000]
mpjan
  • 1,790
  • 5
  • 18
  • 21

2 Answers2

3

This is the quick and dirty solution I implemented:

import urllib
import json

artist = "The Beatles"
songs = ["Love Me Do", "P. S. I Love You", "Please Please Me"]
query = "http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=%s"

google_hits = list()
for song in songs:
    results = urllib.urlopen(query % (artist + " " + song))
    json_res = json.loads(results.read())
    google_hits.append(int(json_res['responseData']['cursor']['estimatedResultCount']))

print google_hits

I'm using the open Google Search API, which is deprecated, but provided that you get yourself an API key for the new Custom Search API, the process should be similar. Also, a quick note about the estimatedResultCount from the documentation:

.estimatedResultCount - supplies the estimated number of results that match the current query. Note that this value will not necessarily match the similar value thats visible on the Google.com search properties.

Carl Ekerot
  • 2,078
  • 1
  • 16
  • 10
  • When I use your code I get an error: TypeError: 'NoneType' object has no attribute '__getitem__' – mpjan Nov 28 '12 at 00:41
  • Code works when I've tried it, but I would suggest you to try implement a similar solution using the non-depricated API. Do you get any response from the call to `urllib.urlopen(...)`? – Carl Ekerot Nov 28 '12 at 02:11
  • Ok, now it seems to be working, but the number of hits retrieved is very different from the one I get when I actually search for it on Google... – mpjan Nov 28 '12 at 14:24
0

You may find xgoogle useful for writing this code: https://github.com/pkrumins/xgoogle

Here is some working code showing how to iterate over and count results of a search: How to use xgoogle

Community
  • 1
  • 1
piokuc
  • 25,594
  • 11
  • 72
  • 102