6

I have several machines each with 128 GB of ram, each host is running a single instance of Elasticsearch. I would like to run another data node on each host and allocate around 30 GB to the jvm heap.

I know I have to create a separate config file .yml and data directory..etc. My question is do I need to modify the service wrapper so that each node will be started/ stopped seperatly?

I am running ES version 1.3 on Centos 6.5

thank you

Yasir
  • 4,567
  • 7
  • 22
  • 19
  • Why do this instead of running bigger single ES JVMs per host? – Matt Ball Feb 12 '15 at 16:21
  • 4
    Because of [this](http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/heap-sizing.html#compressed_oops), @MattBall. – Andrei Stefan Feb 12 '15 at 16:23
  • 2
    jvm Garbage collection is more efficient when heap is < 32 GB – Yasir Feb 12 '15 at 16:25
  • By service wrapper do you mean the init.d script? if so, yes it makes sense to have one init.d script for each node - duplicate the file for each node and modify the sysconfig, lock file and pid lines. – Olly Cruickshank Feb 12 '15 at 16:35
  • Ah – your question didn't initially include the 128 GB figure. – Matt Ball Feb 12 '15 at 16:37
  • possible duplicate of [Multiple nodes in ElasticSearch](http://stackoverflow.com/questions/13477303/multiple-nodes-in-elasticsearch) – Paul Sanwald Feb 12 '15 at 16:39
  • @Olly Cruickshank I currently use sudo service elasticsearch start , which starts the first instance, I wanted to know whether I will need somthing like sudo service elasticsearch2 start to start the second node. – Yasir Feb 12 '15 at 16:46
  • thanks Paul Sanwald that is a good link, but it does not show how to start the different instances as a service. it shows how to run each node manually. – Yasir Feb 12 '15 at 16:47
  • 2
    @Yasir - you'll need to copy `/etc/init.d/elastcsearch` to `/etc/init.d/elasticsearch2` to set up the additional service and modify the file as I described. Also lookup **chkconfig** to see how to automatically start the service when your server starts. – Olly Cruickshank Feb 12 '15 at 17:01
  • thanks Olly, I have got the file in /etc/rc.d/init.d/elasticsearch but not sure where the parameters are passed in for the es_config, es_home etc on this line; `daemon --user $ES_USER --pidfile $pidfile $exec -p $pidfile -d -Des.default.path.home=$ES_HOME -Des.default.path.logs=$LOG_DIR -Des.default.path.data=$DATA_DIR -Des.default.path.work=$WORK_DIR -Des.default.path.conf=$CONF_DIR` – Yasir Feb 12 '15 at 17:17

2 Answers2

10

You need to prepare two elasticsearch.yml config files to configure settings accordingly and specify these files when startup up the two nodes.

bin/elasticsearch -Des.config=$ES_HOME/config/elasticsearch.1.yml
bin/elasticsearch -Des.config=$ES_HOME/config/elasticsearch.2.yml

At least the following should be set differently for the two nodes:

http.port
transport.tcp.port
path_data
path_logs
path_pid
node.name

The following needs to point to the other in both files to allow the nodes to find each other:

discovery.zen.ping.unicast.hosts: '127.0.0.1:9302'

EDIT: the property is now deprecated, look at : https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery-settings.html

See this blog and this discussion

M4rk
  • 2,172
  • 5
  • 36
  • 70
centic
  • 15,565
  • 9
  • 68
  • 125
0

elasicsearch.yml-1

cluster.name: test
node.name: node-1
path.data: /Users/musab/Desktop/elasticsearch/data
path.logs: /Users/musab/Desktop/elasticsearch/logs
node.max_local_storage_nodes: 4

elasicsearch.yml-2

cluster.name: test
node.name: node-1
path.data: /Users/musab/Desktop/elasticsearch/data
path.logs: /Users/musab/Desktop/elasticsearch/logs
node.max_local_storage_nodes: 4

enter image description here

Musab Dogan
  • 1,811
  • 1
  • 6
  • 8