0

I am working on a java app to create an elasticSearch Index. The index is currently built on a dev machine on a weekly basis and the cluster manually copied over the existing cluster on other (test & prod) machines. The new scenario still involves rebuilding the index from scratch each week.

I am working on an update using 5.6.3 and the low & high level rest clients. After some initial investigations (and getting quite a bit of help) I am aiming to go for a 2 (or 3) stage development using aliases.

Initially I will alias the existing index(es) on dev, write all new data to the alias and then replace the index with the alias. The other alternatives to this that I could think of were

  1. Recreating the cluster each time, which I thought would always be a fairly manual process, or

  2. Recreating the indexes and moving them over. As indexes can't be renamed (or so I believe, this would always be more clunky as it would involve creating the indexes on one machine, deleting them from the live machine and then reading them. Currently we have one node in the cluster and so this would involve a fair amount of down time (although if the aliases turn out to be a bad solution I guess we could add another node and replace the index one node at a time).

so I have opted for

  1. using aliases. This way I can start by just recreating the index and copying it over (phase 1) and then move to a more sophisticated solution to restore the data programmatically & without bringing down elasticSearch (phase 3 is the dream of just updating the index if we ever get our data sorted properly!)

So down to the question. I really would like to create an empty alias of the index straight off. As far as I can see this is not possible. Among other pages this github issue., although from an older version, makes it appear that a move like this would be counter-productive and is not something elastic want to introduce. So I was just wondering if

  1. I am looking at the wrong design entirely or
  2. there is now a way to create an empty alias or
  3. whether I should create an alias and then delete all documents from the alias instead ?
gringogordo
  • 1,990
  • 6
  • 24
  • 60
  • 1
    I am not sure what will an empty alias buy you. You can always work with a single alias. Live application will point to the alias name. The load process would create a new index and then you can switch the alias from old index to new index. Individually deleting all documents is not an good idea. Instead you can drop the index. Also, you can explore Elasticsearch Curator for snapshot/restore of indexes. – askids Oct 19 '17 at 17:54
  • Okay so I think I misunderstood. I thought an alias was part of the metadata of an index and so you couldn't swap the alias over with another index. I was thinking of pointing the app to an index and creating temporary aliases to rebuild the index and then swap the alias back in. ACTUALLY the alias can be pointed to a different index and so it makes more sense to do the opposite and set the application to look at an alias ([appind]) and then point this to indexes - which change weekly - called appind[ddmmyyyyhhmm]. (Is this correct!?) – gringogordo Oct 19 '17 at 18:18
  • 1
    Correct. The application need not worry about whether the data is from 19th or 20th. It will simply point to alias appind. Once your data restore/load is completed, you can swap the alias with single post. You can have a remove action and add action in the same request to swap the alias from old index to new index. – askids Oct 19 '17 at 18:33

1 Answers1

1

"Added comment as answer": Correct. The application need not worry about whether the data is from 19th or 20th. Application can simply point to alias appind. Once your data restore/load is completed, you can swap the alias with single POST request from old index to new index. You can have a remove action and add action in the same request to swap the alias from old index to new index.

askids
  • 1,406
  • 1
  • 15
  • 32