1

I have a python program that takes the md5 & sha1 hash values of passwords and searches for them on the internet using Google's custom search api. The problem is that I'm getting 0 results(which means the hash probably isn't in a rainbow table) when I run the program. But when I searched using my browser, I get a whole bunch of results, in fact at least 10 pages of results.

Could the problem lie in the cx value I used? I picked it up from the sample program provided by google as I couldn't figure out how to get one for myself. Or does the custom search api give only selected results and it's futile trying to get more results from it?

marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
elssar
  • 5,651
  • 7
  • 46
  • 71

2 Answers2

2

I know it's pretty old post but it is still returned very high in google results so a little bit of clarification:

  1. You can create your own CSE in here: https://www.google.com/cse/ .
  2. API codes can be created using API console: https://cloud.google.com/ .
  3. Using Google Custom Search you can search the whole Web: go to the system from point 1, from the menu on the left choose the CSE to edit, then in the Configuration -> Basics -> Sites select the option to search the whole Web and finally remove previously specified sites.
  4. Still using CSE you might not get the same results as using live google as it does not include google features (real-time results, social features etc.) and once you specify more than 10 sites to look on it can actually use sub-index. More information can be found in here: https://support.google.com/customsearch/answer/70392?hl=en
Szpilona
  • 121
  • 2
  • 9
  • Despite completing all the steps and reading docs above, I get completely different top results from python code vs using the browser. I am getting results from entire internet, but not even close to the same ones as from the browser. Any more things to check? – Ville M Sep 30 '18 at 20:52
1

The Google Custom Search API let's you search the Google indexes for a specific website only, and you will not find any results from anywhere else on the internet. The cx parameter tells Google what website you want to search.

From the Google Custom Search Engine page:

With Google Custom Search, add a search box to your homepage to help people find what they need on your website.

You could use the deprecated Google Web Search API (JavaScript API, should work until November 2013), or you'd have to scrape the HTML UI provided to your browser instead (also see What are the alternatives now that the Google web search API has been deprecated?).

Community
  • 1
  • 1
Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343