0

I'm requested to write an inverted index, so I would like as a start to write a java program which google searches a word and putting the results into an arraylist.

Here's my code:

String search = "Dan";
String google = "http://www.google.com/cse/publicurl?cx=012216303403008813404:kcqyeryhhm8&q=" + search;
URL url = new URL(google);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/json");
BufferedReader reader = new BufferedReader(new InputStreamReader(
    (conn.getInputStream())));
// Gather the results to a String array
List<String> resultsList = new ArrayList<String>();
String r;
while ((r = reader.readLine()) != null)
    resultsList.add(r);
conn.disconnect();
System.out.println("Google Search for: " + search + " Is Done!");

The programs runs with no crashes in the middle, but I get only a source code of a page (which does not contain any links).

What do I need to change in the code? Maybe I need a whole different method?

Andrew Thompson
  • 168,117
  • 40
  • 217
  • 433
DanielY
  • 1,141
  • 30
  • 58
  • Any other suggestions on where to search from in a java program, @AndrewThompson? – DanielY Jun 18 '13 at 14:17
  • 1
    Instead of doing a google search, i suspect you want (or should) in the end use a crawler for crawling your web pages and building your index. One library that supports writing a crawler is Crawler4J. Also, just copying the google inverted index really doesn't seem like a good approach. For testing, you can start by indexing single web pages without a crawler. – kutschkem Jun 18 '13 at 14:36

1 Answers1

2

If you want to use google search in your app you should use Google's API for that:

Custom search API

You get search results in JSON format.

juniperi
  • 3,723
  • 1
  • 23
  • 30
  • What does that mean "JSON Format"? Is there any way to work with it in Java? (For instance, collect the resulting url's to a String list?) – DanielY Jun 18 '13 at 14:26
  • +1 - I removed my earlier comment, as this answer showed it to be noise. – Andrew Thompson Jun 18 '13 at 14:36
  • @user1067083 http://stackoverflow.com/questions/1688099/converting-json-to-java Maybe this helps – juniperi Jun 18 '13 at 14:37
  • 2
    @user1067083 JSON is a data format. In Java, you can easily work with it for example with the GSON library. – kutschkem Jun 18 '13 at 14:38
  • @user1067083 Sorry but never used that API. Here you can find more information about results: https://developers.google.com/custom-search/v1/using_rest#WorkingResults – juniperi Jun 18 '13 at 18:51