1

I have a web application which uses Lucene.Net and NHibernate.Search as a full-text search engine. NHibernate.Search is setted up such that whenever a change is done in the database, it is propagated to the Lucene index.

The web application is running using 4 worker processes. First of all, is this an issue? I am noticing that the Lucene index is not 100% in sync with the database. Some changes to the database are not appearing in the Lucene index. However, when I manually try to re-index the data, this works fine.

Are there any concurrency implications with using Lucene.Net in a multi-process environment?

Karl Cassar
  • 6,043
  • 10
  • 47
  • 84
  • You should also remember that the IndexReader only works on the index as it existed when it was opened -- if the index was updated after the IndexReader was opened, you won't see those new documents. – Mark Leighton Fisher Nov 07 '12 at 23:49
  • @MarkLeightonFisher The issue was due to `LockObtainTimeouts` - First of all I had NHibernate loggers switched off, and hence couldn't detect it. The issue was that when such a timeout is triggered, the item is not saved. I have still to find the exact cause of all these timeouts. – Karl Cassar Nov 08 '12 at 09:25
  • Did you have any luck with this Karl? – DavidWainwright Jan 14 '13 at 11:04
  • @coalvilledave - No, unfortunately not yet! Randomly, the processes will crash when using the same Lucene index, or lock indefinitely. In order to circumvent this, I've added a backbround process which monitors the index for any locks which have been inplace excessively, and automatically delete them. I am not proud of this solution, but it works and don't have the time to re-work the system to use SOLR, as it seems should be the best way, or splitting out the search to an external process. – Karl Cassar Jan 14 '13 at 11:46

1 Answers1

0

This sounds like an issue with an open IndexWriter keeping locks on the index directory. One worker process would lock the index for the other processes.

Lucene.Net can be used in a multi-process environment as long as there's only one writer per index. Different directory implementations enforce this in different ways, usually involving a file named write.lock.

A common solution is to have a separate search process which handles indexing and searching.

sisve
  • 19,501
  • 3
  • 53
  • 95
  • Since then, I've moved to using a document-database over Lucene just for searching as the search functionality is extracted out to an external process. From this, I learnt it the hard-way that Lucene only supports one writer per index. – Karl Cassar May 07 '13 at 11:40