1

How do I write a Python program that interacts with the browser? I.e. I want to write a program that types (pastes) strings form a dictionary (or a list) into Google and if the result is not "0 matches found" then copies the URL of the (say) first page that Google finds (into a text file). My main problem is that I don't know how to make python do things within the browser. (how to make it paste the strings, "press search", see whether there are any results...)

DNA
  • 42,007
  • 12
  • 107
  • 146
Nesa
  • 2,895
  • 2
  • 12
  • 19
  • 4
    This question needs more help than we can provide. We like helping people, but sometimes the person needs to help themselves first by reading a book on the language, the on-line documentation, or asking someone they know who can help them. SO is more for specific problems you have with the code you've already written. – Prune Feb 15 '16 at 21:44
  • 1
    I believe [PhantomJS + Selenium](http://stackoverflow.com/questions/13287490/is-there-a-way-to-use-phantomjs-in-python) is what you want to use – OneCricketeer Feb 15 '16 at 22:21
  • @Prune In general I think you are right but in this particular case the request module given in the answer solved my problem. – Nesa Feb 16 '16 at 10:04
  • You have an up-vote and two down-votes, none of which is me. Please note that getting an answer does not make your question good. The purpose of the suspension system is to give a "time-out" to people who violate the guidelines. – Prune Feb 16 '16 at 17:39
  • Read the help documentation. If you don't understand the overall purpose or implementation, please take the discussion to the Meta site. These issues don't belong on a question page. – Prune Feb 16 '16 at 19:09
  • @Prune I read the guidelines and you are right didn't go by them when I came stackoverflow having not searched for the answer elsewhere, but your username is obnoxious, I think you should change it – Nesa Feb 16 '16 at 21:46
  • How is my user name obnoxious? It's the name of a dried fruit. – Prune Feb 16 '16 at 21:48

2 Answers2

3

You don't need to scrape. Google provides a search api that you can use. You can do something like this:

import requests

api_url = 'http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=%s'
query = 'xxx'
query_url = api_url %query
response = requests.get(query_url).json()
results = response['responseData']['results']
if results:
   first_result_url = results[0]['url']
Walid Saad
  • 951
  • 1
  • 8
  • 14
2

The Ajax API is no longer available and the response directs you to use Google's Custom Search API.

{"responseData": null, "responseDetails": "The Google Web Search API is no longer available. Please migrate to the Google Custom Search API (https://developers.google.com/custom-search/)", "responseStatus": 403}
dkm
  • 101
  • 1
  • 6