2

I am looking for some benchmarking of how many number of request can be indexed per second by ElasticSearch server. Surely, it would be subject to many factors like analyzers used, capacity of ES nodes, number of nodes in cluster and so on.

I am assuming document with approximately 20 fields, all "not_analyzed" and running on single node with i7 and dedicated 8GB heap memory.

  1. What is predicted or benchmarked approx. number of requests handled per second by ElasticSearch server?
  2. Say it is x req/sec in question 1. Then how does this capacity increase as we keep adding another node with same config in the cluster. Is it linear like 2x, 3x & 4x as we add 2, 3 or 4 nodes?
Vishal Shukla
  • 2,848
  • 2
  • 17
  • 19

1 Answers1

2

With the rig you describe and your particular document structure, I believe you'll be able to get 5k-15k per second of documents. The parameters affecting this number are numerous, including:

1. refresh_interval - set it to something other than 1s (for example 30s, or 60s).
2. Sharding policy
3. usage of SSD.
4. "_all" field.
5. usage of bulk api with the "right" number of index requests.
6. indexing without document ids.
7. translog settings
...

The increase of indexing throughput should be linear to the number of nodes you have. Meaning, with two identical machines of your description, you should get twice the throughput.

Here's a link to a question I asked a month ago which might assist you further.

Community
  • 1
  • 1
Roman
  • 2,108
  • 1
  • 18
  • 20