search in google using python

Question

I'm looking for any python API to search in google (normal search), and I found only the following code:

import pprint
from googleapiclient.discovery import build

    def google_search(search_term, api_key, cse_id, **kwargs):
        service = build("customsearch", "v1", developerKey=api_key)
        res = service.cse().list(q=search_term, cx=cse_id, **kwargs).execute()
        return res['items']

my_api_key = "Google_API_Key"
my_cse_id = "my_cse_id"
results = google_search('stackoverflow site:en.wikipedia.org', my_api_key, my_cse_id, num=10)
for result in results:
      pprint.pprint(result)

The first thing is that it generate this error, I tried to resolve but no chance :/ : results

= self.google_searchS('stackoverflow wikipedia.org', my_api_key, my_cse_id)

TypeError: google_searchS() takes 3 positional arguments but 4 were given

The other thing is there any other API for the normal search "not custom search" .. ?

why you are calling google_search() function by self instance, as per function definition no need to call by using self instance. There are two way: 1. If you want to call in that way then you have to define method decorator @static method 2. Don't call function using self instance if function in out of class scope. — Anup, Feb 26 '18 at 18:29
I edit the code, it was with self instance because i copied it from a class — Minions, Feb 26 '18 at 18:31

score 1 · Answer 1 · edited Feb 05 '20 at 15:27

In my case I use python3.7. I've also enabled Search the entire web in my search engine:

And changed that code as follows:

from googleapiclient.discovery import build
import pprint

my_api_key = "ASDFASDF4TzB-FASDGDFG9n3Wfdsdffasd3PQ"
my_cse_id = "2199288337529387487289738:8asfiejkfjke"

def google_search(search_term, api_key, cse_id, **kwargs):
    service = build("customsearch", "v1", developerKey=api_key)
    res = service.cse().list(q=search_term, cx=cse_id, **kwargs).execute()
    return res['items']

results = google_search(
    'open world', my_api_key, my_cse_id, num=10)
for result in results:
    pprint.pprint(result)

And it outputs 10 results like this:

Note: I've obfuscated my api_key and cse_id before posting. Also, I've configured venv before as it is described here.

score 0 · Answer 2 · answered Feb 27 '18 at 12:50

I found the solution, the problem is in class calling .. anyway, it did not help me, why? Google custom search API is just created to add a searching option in sites, and it should be customized .. also it doesn't retrieve results as normal searching ..

But to use the "normal" Google search engine, I got this code:

def searchGo(self,search_term, number_results, language_code):
        USER_AGENT = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36'}
        assert isinstance(search_term, str), 'Search term must be a string'
        assert isinstance(number_results, int), 'Number of results must be an integer'
        escaped_search_term = search_term.replace(' ', '+')
        google_url = 'https://www.google.com/search?q={}&num={}&hl={}'.format(escaped_search_term, number_results, language_code)
        response = requests.get(google_url, headers=USER_AGENT)
        response.raise_for_status()
        # Parsing
        soup = BeautifulSoup(response.text, 'html.parser')
        found_results = []
        rank = 1
        result_block = soup.find_all('div', attrs={'class': 'g'})
        for result in result_block:
            link = result.find('a', href=True)
            title = result.find('h3', attrs={'class': 'r'})
            description = result.find('span', attrs={'class': 'st'})
            if link and title:
                link = link['href']
                title = title.get_text()
                if description:
                    description = description.get_text()
                if link != '#':
                    found_results.append({'link': link,'keyword': search_term, 'rank': rank, 'title': title, 'description': description})
                    rank += 1
        return found_results

and you can call it like that:

str1 = ts.searchGo('obama spent two millions for Egyption', 10, 'en')
    for i in range(len(str1)): print(str(str1[i]['rank']) + "\n" + str(str1[i]['title']) + "\n" + str(str1[i]['description']) + "\n")

search in google using python

2 Answers2