3

I would like to download usenet messages to my personal Mysql database, so without using any newsreader. At the moment i'm using IpWorks component for .NET . I've created a simple app in Vb.Net, It works, but it's realy slow... to download and to do the INSERT INTO . Anyone know some better method to "massive" download message and insert it into a simple mysql table?

casperOne
  • 73,706
  • 19
  • 184
  • 253
stighy
  • 7,260
  • 25
  • 97
  • 157
  • if you're willing to try a different OS, this problem has been largely solved with [leafnode](http://leafnode.sourceforge.net/), which is amazingly easy to use. It will at least solve the downloading portion. Why do you want to use a database? – Thufir May 07 '12 at 01:37

1 Answers1

1

You might want to try using a bulk insert or a delayed insert instead of entering items row by row sequentially.

Additionally, a lot of this can be done asynchronously. Most of the work you are doing (writing to the database and downloading the content from nntp) is not bound to resources on your machine. You can perform the downloading and reading of the nntp content in a thread separate from the ones that perform the inserts (you should buffer the content you want to insert before you send it to mysql).

Basically, you want a producer consumer pattern.

casperOne
  • 73,706
  • 19
  • 184
  • 253