0

I'm still trying to understand how elasticsearch is setup. We have 4 EC2 servers and a loadbalancer, but looking at the elasticsearch _head plugin, the shards look to be scattered a bit. Is this common, or is there something I'm missing that I need to look into?

elastic shards

EDIT:

as javanna said below, this is just default setup. He explains it in amazing detail here in this post

Community
  • 1
  • 1
jeremywoertink
  • 2,281
  • 1
  • 23
  • 29
  • Looks good, why are you asking? What would you expect instead? – javanna Oct 14 '13 at 08:29
  • I was expecting all 4 machines to have all 4 shards. My search is really slow, and there's not a lot of traffic. I would expect 4 AWS instances to be able to handle it, but it's not. Understanding that this setup is not an error allows me to focus in other areas :) Thanks for the information. – jeremywoertink Oct 15 '13 at 15:59
  • You have a total of 10 shards to be distributed over 4 machines. How can they have 4 shards each? – javanna Oct 15 '13 at 16:54
  • Is that 10 a limit of elasticsearch? Does that mean if I add 2 more machines, I'll have 10 shards to distribute between 6 machines? Can I increase that shard limit, and is that the correct thing to do? – jeremywoertink Oct 15 '13 at 18:39
  • 1
    You can configure the number of shards when creating the index, otherwise it will use defaults (5 shards, 1 replica). Didn't realize you were missing this bit. Maybe this one is interesting too: http://stackoverflow.com/questions/15694724/shards-and-replicas-in-elasticsearch – javanna Oct 16 '13 at 08:05

1 Answers1

0

Looks like you have four shards for your index and two replicas for each shard spread over four machines. Each shard exists on two machines. Some machines have more shards than others. So, seems fine to me and it looks like you are using the default settings. You will probably want to read up on the configuration options for this.

Jilles van Gurp
  • 7,927
  • 4
  • 38
  • 46
  • Ok, but you think this setup is fine? Or is your suggestion to not use the default config, and have all 4 shards on each machine? – jeremywoertink Oct 13 '13 at 20:38
  • 1
    Impossible to tell without more information about your writing and reading requirements. As a rule of thumb, more machines and shards increase your write throughput; more replicas and machines give you more read bandwidth. – Jilles van Gurp Oct 14 '13 at 07:58