3

I have a cluster containing 7 nodes, I've recently noticed that we are only using 5 primary shards, meaning that per index - we are only utilizing 5 nodes out of 7. I would like to add two additional primary shards for newly created indices.

I've added the following to elasticsearch.yml and restarted the cluster:

index.number_of_shards: 7 index.number_of_replicas: 0

However, it's been a few days and a few new indices since - and I still see its sharding to 5 pieces.. can anyone explain why? am I missing some additional configuration here?

Thanks!

Eytan Avisror
  • 2,860
  • 19
  • 26

2 Answers2

2

You have to reindex affected index. As I imagine there is only 1 index in the cluster. Number of shards can not change automatically in an existing index.

This relative question might help and this question if you do not want any downtime, it works with aliases and basically 2 indexes- 1 before resharding and 1 afterwards and then you switch aliases to point in wherever index you want

Community
  • 1
  • 1
dimzak
  • 2,511
  • 8
  • 38
  • 51
  • Guys - Thanks for the comments but I think you misunderstand, I have new indices created every day at midnight, i just want new indices to be created with more shards. old indices will eventually phase out. is that possible? – Eytan Avisror May 12 '15 at 08:38
  • @user3352229 Can you post `GET /yourdailyindices/_settings` for indices before you change shard number and after? – dimzak May 12 '15 at 08:55
  • here is the settings of an index that was created after i changed the setting and restarted the cluster: { "2015-05-12" : { "settings" : { "index" : { "creation_date" : "1431388790550", "uuid" : "H5bzXRekT0WDni2BEDJsfA", "number_of_replicas" : "0", "number_of_shards" : "5", "version" : { "created" : "1040499" } } } } } – Eytan Avisror May 12 '15 at 09:20
  • its still creating indices with default value of 5 primary shards, looks like its ignoring the setting in elasticsearch.yml for index.number_of_shards: 7 – Eytan Avisror May 12 '15 at 09:21
  • 1
    @user3352229 Did you restarted nodes after the change in elasticsearch.yml? I find it more useful to use [templates](http://www.elastic.co/guide/en/elasticsearch/reference/1.3/indices-templates.html) for your case, it's easier to change settings and you don't have to mess with elasticsearch.yml – dimzak May 12 '15 at 09:56
  • yes i did, i restarted the service on each node in the cluster. I think I found the issue, I found a template "default" with the following: GET /_templates/default { "default" : { "order" : 0, "template" : "*", "settings" : { "index.number_of_replicas" : "0", "index.number_of_shards" : "5" }, "mappings" : { }, "aliases" : { } } } When I try to PUT same json with different number of shards I get: PUT /_templates/default { "error" : "ActionRequestValidationException[Validation Failed: 1: template is missing;]", "status" : 400 } – Eytan Avisror May 12 '15 at 10:13
  • nvm, I was able to make the change! Thanks very much! – Eytan Avisror May 12 '15 at 10:33
  • @user3352229 template was the problem? – dimzak May 12 '15 at 10:34
  • Yes -- looks like template settings override global settings in elasticsearch.yml – Eytan Avisror May 12 '15 at 11:44
1

You cannot add primary shards without creating a new index and re-indexing all your data.

You can add replicas, but not primaries without reindexing. For a zero-downtime re-indexing scenario, check this link.

Andrei Stefan
  • 51,654
  • 6
  • 98
  • 89