I wrote: http://arachnode.net
"An Open Source C# web crawler with Lucene.NET search using SQL 2005/2008/CE"