1

Right now I have a program with a Gui that indexes a url that I specify. I need to index 15 things at a time and I have been just opening 15 windows of the program and individually inputting the urls that i want to index. However these 15 url's change every hour or so... I have a separate program that stores these 15 constantly changing urls in a table in my mysql database. I am able to fetch these urls from my database(I store them in an arraylist) but im not sure how to go about multi-threading my application so that i dont have to do the manual work of inputing the urls into my application.

My question: Can someone give me an example/link me to a tutorial of how I would go about creating a new thread for each url in my arraylist(also this arraylist will change so will i need to make a new thread for changing this arraylist aswell?)

Ive looked at the java site on concurrency and high level concurrency but didn't really understand the examples they gave(I am still a beginning programmer so please bear with me)

Hopefully I explained what im trying to do with enough detail Thanks in advance

EDIT: The urls i index change every couple of seconds which is why i dont think i can go through my method with each url one after another, hence why i believe it needs to be multithreaded?

2nd EDIT(I believe these guys understand what im asking):

@Jon Storm There are two issues going on 1) Getting URL list 2) Accessing said URLs. I would make the URL fetcher single-threaded and then dispatch out to a Thread Pool of fetchers. This dispatcher can also queue pending fetches, etc. – pst

@Jon Storm: could you please update your question to describe what you want to do more explicitely? If In understood correctly, you want to index a list of 15 URLs again and again, because the contents of the pages at these URLs changes every 3 seconds. And you want to update the list of URLs to index every hour, by getting them from a database. Is that right? – JB Nizet

Jon Storm
  • 185
  • 1
  • 3
  • 10

2 Answers2

2

It seems to me that your problem is not with multi-threading, but with inputting something in a GUI from the application fetching the URLs from the database.

Why don't you simply reuse the class (or some of the code, if it's impossible to reuse the classes as is) of your GUI application (i.e. the URL indexing method) inside the application which fetches the URLs from the database.

My guess is that you could very well index these 15 URLs one after the other, in a single thread. I would try doing that before trying to use threads.

The program would look like this:

  1. Fetch the 15 URLs from the database and put them into a List
  2. Iterate through the list and index each URL
  3. Sleep for some time,
  4. Go to 1

EDIT:

Since it seems the URLs must be indexed again and again until the list of URLs changes, I would use this algorithm:

  1. Create a thread pool using Executors.newCachedThreadPool()
  2. Get the URLs from the database
  3. For each URL, create a task which will index the URL again and again, until interrupted (check that Thread.interrupted() returns false at each iteration)
  4. submit each task to the executorService created at step 1, and keep the returned Future inside a list
  5. Sleep/wait until the list of URLs to index changes
  6. Cancel each Future (cancel(true)) of the list of Future instances
  7. Go to step 2.
JB Nizet
  • 678,734
  • 91
  • 1,224
  • 1,255
  • Sorry I forgot to mention the key part of what I'm doing... These urls that I index change every couple of seconds or so which is why i believe I need to multi thread it otherwise the information might change on a page while im indexing another and i need to index the site every time it changes Unless-do you think i could index through 15 sites in under 3 seconds total? If so i could do what your suggesting, but I dont believer the way im doing it that it would be possible – Jon Storm Jul 03 '11 at 19:24
  • @Jon Storm There are two issues going on 1) Getting URL list 2) Accessing said URLs. I would make the URL fetcher single-threaded and then dispatch out to a Thread Pool of fetchers. This dispatcher can also queue pending fetches, etc. –  Jul 03 '11 at 19:29
  • @Jon Storm: could you please update your question to describe what you want to do more explicitely? If In understood correctly, you want to index a list of 15 URLs again and again, because the contents of the pages at these URLs changes every 3 seconds. And you want to update the list of URLs to index every hour, by getting them from a database. Is that right? – JB Nizet Jul 03 '11 at 19:49
  • @pst I believe that is what i want to do, im just not sure about how to go about doing that, any links to tutorials or examples would be greatly appreciated. – Jon Storm Jul 03 '11 at 21:52
0

SwingWorker, shown here, is a good choice.

Community
  • 1
  • 1
trashgod
  • 203,806
  • 29
  • 246
  • 1,045