Running Sunspot in production

Question

I am running Rails 3.1 in production on Ubuntu 10.4 LTS, and with mongoid. I also install tomcat6 and solr via this guide. Everything works great, but I am not sure if I have it configured the best. I disable auto_commit_after_request in my sunspot.yml ask talked about here.

I also uncommented the auto-commit section in my solrcongif.xml. It just seems like updates to indexed records take a little longer than updates to non-indexed records. Is there any way I can check to see if auto_commit_after_request is really turned off?

Also, when I check the analysis page in solr, models that have been updated appear to be doubled indexed (I can tell because there isn't a lot of data yet). So I have one model with a keyword, but in the index it says there are 3 instances of the word (one for the create and two updates I did on the mondel). Is that normal? It just seems like this would create a bias towards frequently updated models, but also increase the size of the index.

Below is my config/sunspot.yml file. If needed I can post the solrconfig.xml too, but it is big so I will leave it out for now. Is there any guide on tuning sunspot for production? Thanks!

production:
  solr:
    hostname: localhost
    port: 8080
    log_level: WARNING
    path: '/solr'
    auto_commit_after_request: false

development:
  solr:
    hostname: localhost
    port: 8982
    log_level: INFO

test:
  solr:
    hostname: localhost
    port: 8981
    log_level: WARNING

Answer 1

There's a whole bunch of questions here.

Most of your performance tuning will be on SOlr rather than Sunspot which is just an interface to Solr. See http://wiki.apache.org/solr/SolrPerformanceFactors. As you've noted when you commit and whether it is done in the background can affect performance of your app.

Updates in Solr (Lucene) actually work by removing the old object and adding it in again. This might be the cause of your speed differences noticed.

Updating won't add the terms to the index multiple times. You must be observing something else. If you've used the same index for development sometimes you might see old field names polluting the index; try deleting the whole index (the actual index folder on disk) and starting again;