0

We are looking to have a Solr 4.9 setup where we have a very simple crawler wipe out and load up a "crawler" core then trigger a copy of the data over to "search" core when the crawl is done. The purpose of this is that our crawler is VERY simple, and doesn't really track documents in a way that would be conducive to doing updates and deleted. Basically, the crawler will be wiping out the entire "crawler" core, ripping though about 50k documents (committing ever 1000 or so), and then trigger something to copy over the data to the other "search" core.

Assuming we would have to restart the Search core, how could this be made possible from command-line or code?

CodeChimp
  • 8,016
  • 5
  • 41
  • 79

1 Answers1

1

Create a third core as a copy of the search core. Then use the mergeindexes command in CoreAdmin to merge two different cores into the third one. After the merge finishes, swap the the third core with the old search core. Then UNLOAD the swapped out core (with deleteInstanceDir=true if you're feeling that you can permanently remove the old data).

Something like:

http://localhost:8983/solr/admin/cores/action=CREATE&name=core0&instanceDir=path_to_instance_directory&config=config_file_name.xml&schema=schema_file_name.xml&dataDir=data

http://localhost:8983/solr/admin/cores?action=mergeindexes&core=core0&indexDir=/opt/solr/crawl/data/index&indexDir=/opt/solr/index/data/index

http://localhost:8983/solr/admin/cores?action=SWAP&core=search&other=core0

http://localhost:8983/solr/admin/cores?action=UNLOAD&core=core0
MatsLindh
  • 49,529
  • 4
  • 53
  • 84
  • I don't think we need the merge part, but the `SWAP` is very interesting. Is it possible to do this using Solrj, or do you have to go through the CoreAdminHandler using HTTP? – CodeChimp Aug 26 '14 at 11:30
  • If you don't want to merge it into existing data, then just skipping the merge part would be OK. See the response to http://stackoverflow.com/questions/6668534/how-can-i-get-a-list-of-all-the-cores-in-a-solr-server-using-solrj for how to use SolrJ for CoreAdmin. – MatsLindh Aug 26 '14 at 12:18