Google Search with Python

Question

From this question I've learned how to google-search using Python 3. But in the example given there you can retrieve only the first 4 results. I need to retrieve information about the first 25 results (at least).

For each result I want to be able to get its:

• title
• url
• visible url (the visible url of http://en.wikipedia.org/wiki/Information_retrieval is http://en.wikipedia.org/)
• site description

How can I do it?

EDIT: I'm using Python 3.1

Answer 1

This thread should tell you what you need to know. In short, add &rsz=large to the URI to get eight results at a time, and &start=8 (or 16, 24, etc.) gets you results starting with the indicated number. In no case can you get more than 64, and some searches allow only 8.

Answer 2

Use the JSON/Atom Custom Search API and if necessary call it multiple times by using the start parameter to receive all results you are interested in.