what are the steps to make a word search for a website?

Question

I want to write a word search,which connects to a specific website(huge one),takes the word from user,searches the site and returns the strings which contain the word;this should be written in java and as an applet.I have read some tutorials and questions on this,and understood what have to be done is:

1.connect to a website and get the content of a website and save it to a string.(this should be done with a webcrawler which will be made from my own code for connecting to website and save the content to a string + jsoup library to parse the html code).

2.save the datas to a database(in my case nosql database).

3.index the datas in database.

4.query the database to show the results.

5.make a UI for showing the search results(I use swing.japplet).

now my qustions are:

1.have I understood correctly the steps which I have to go?(please explain me in details if a step is unnecessary or necessary)

2.Is it necessary to have a database?

notice:I want to implement it myself,without using ready things such as lucene,nutch,solr,...

edit:3 people told me applet is not suitable for such a thing,so what should be the replacement?

many many thanks for your help.

score 0 · Answer 1 · answered Oct 05 '13 at 05:52

0

You should look at using Lucene, as it does most of what you want here.

You should not use applets.

answered Oct 05 '13 at 05:52

MeBigFatGuy

28,272
7
61
66

no I want to implement it myself.not by ready things such as lucene,why not applets? – fereshteh Oct 05 '13 at 05:53
@user2841094 - I've explained why applets are a bad idea in my Answer. – Stephen C Oct 05 '13 at 06:03
what should it be instead of an applet?a web service? – fereshteh Oct 05 '13 at 06:05

score 0 · Answer 2 · answered Oct 05 '13 at 05:54

0

For small data set, database should be sufficient. Databases like mysql comes with full text search functions.

For bigger data set, you might want to consider Lucene or Solr.

answered Oct 05 '13 at 05:54

interskh

2,511
4
20
20

you mean I should use mysql instead of nosql databases? – fereshteh Oct 05 '13 at 05:56
if i were you, i will just go with solr. – interskh Oct 05 '13 at 06:01

score 0 · Answer 3 · answered Oct 05 '13 at 05:56

That is one way way to implement this. Another (simpler) way would be to use an existing text search / indexing engine like Lucene / Solr. Going to the effort of reimplementing the "text search / indexing" wheel using database technology strikes me as a waste of effort, unless you have a sound technical reason for doing so.

You do need to has some kind of database, because indexing a website on the fly would simply not work. Lucene will handle that.

I think your choice of Java applets to build the UI is a bad idea. There are other technologies that give results that are as good or better ... without the security risk of a Java browser plugin.

Finally, another way to make your website searchable is to get Google to do it for you. Make your website content indexable, and then use Google's search APIs.

Start by looking at this: http://stackoverflow.com/questions/2055430/list-of-rich-web-application-technologies — Stephen C, Oct 05 '13 at 06:05

what are the steps to make a word search for a website?

3 Answers3