Elasticsearch 2.1.1 on localhost Java client error: MasterNotDiscoveredException[waited for [30s]]

Question

So on my localmachine with latest version of elasticsearch, java client node, doesn't receive any information regarding masternode. And disconnects.

I have tried the following code even with default configuration. And still it doesn't work.

If I use TransportClient everything works fine.

Anyone knows what's going on?

Java Client Code

Node node = nodeBuilder()
                .clusterName("Scrapper")
                .client(true)
                .data(false)
                .local(false)
                .settings(
                    Settings.builder()
                        .put("path.home", ".")
                        .put("name", "JG")
                        .put("http.enabled", false)
                        .put("node.master", false)
                        .put("discovery.zen.ping.multicast.enabled", false)
                        .put("discovery.zen.ping.unicast.hosts", "127.0.0.1:9300")
                        .put("network.host", "127.0.0.1")
                        .put("transport.tcp.port", "9301")
                )
                .node();
Client client = node.client();

System.out.println("Firing Requests.");

String INDEX = "test_index";
boolean indexExists = client.admin().indices().prepareExists(INDEX).execute().actionGet().isExists();
if (!indexExists) {
  client.admin().indices().prepareCreate(INDEX).execute().actionGet();
}

SearchResponse allHits = client.prepareSearch(INDEX)
                             .addFields("title", "category")
                             .setQuery(QueryBuilders.matchAllQuery())
                             .execute().actionGet();

System.out.println("allHits = " + allHits);

Java Client Logs:

Jan 13, 2016 12:28:50 PM org.elasticsearch.node.Node <init>
INFO: [JG] version[2.1.1], pid[33792], build[40e2c53/2015-12-15T13:05:55Z]
Jan 13, 2016 12:28:50 PM org.elasticsearch.node.Node <init>
INFO: [JG] initializing ...
Jan 13, 2016 12:28:50 PM org.elasticsearch.plugins.PluginsService <init>
INFO: [JG] loaded [], sites []
Jan 13, 2016 12:28:52 PM org.elasticsearch.node.Node <init>
INFO: [JG] initialized
Jan 13, 2016 12:28:52 PM org.elasticsearch.node.Node start
INFO: [JG] starting ...
Jan 13, 2016 12:28:52 PM org.elasticsearch.transport.TransportService doStart
INFO: [JG] publish_address {127.0.0.1:9301}, bound_addresses {127.0.0.1:9301}
Jan 13, 2016 12:28:52 PM org.elasticsearch.discovery.DiscoveryService doStart
INFO: [JG] Scrapper/5OJu376pS7Gz4HSsZV-yFA
Firing Requests.
Jan 13, 2016 12:29:22 PM org.elasticsearch.discovery.DiscoveryService waitForInitialState
WARNING: [JG] waited for 30s and no initial state was set by the discovery
Jan 13, 2016 12:29:22 PM org.elasticsearch.node.Node start
INFO: [JG] started
Exception in thread "main" MasterNotDiscoveredException[waited for [30s]]
    at org.elasticsearch.action.support.master.TransportMasterNodeAction$4.onTimeout(TransportMasterNodeAction.java:154)
    at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:239)
    at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:574)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

Elasticsearch Logs:

[2016-01-13 12:28:44,579][INFO ][node                     ] [MasterScrapper] version[2.1.1], pid[33773], build[40e2c53/2015-12-15T13:05:55Z]
[2016-01-13 12:28:44,579][INFO ][node                     ] [MasterScrapper] initializing ...
[2016-01-13 12:28:44,747][INFO ][plugins                  ] [MasterScrapper] loaded [license, marvel-agent], sites []
[2016-01-13 12:28:44,762][INFO ][env                      ] [MasterScrapper] using [1] data paths, mounts [[/ (/dev/disk0s2)]], net usable_space [302.5gb], net total_space [465.1gb], spins? [unknown], types [hfs]
[2016-01-13 12:28:46,023][INFO ][node                     ] [MasterScrapper] initialized
[2016-01-13 12:28:46,024][INFO ][node                     ] [MasterScrapper] starting ...
[2016-01-13 12:28:46,093][INFO ][transport                ] [MasterScrapper] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}
[2016-01-13 12:28:46,099][INFO ][discovery                ] [MasterScrapper] Scrapper/NfXHsmWlTJ23bPz5CJ5D9w
[2016-01-13 12:28:49,115][INFO ][cluster.service          ] [MasterScrapper] new_master {MasterScrapper}{NfXHsmWlTJ23bPz5CJ5D9w}{127.0.0.1}{127.0.0.1:9300}{master=true}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2016-01-13 12:28:49,125][INFO ][http                     ] [MasterScrapper] publish_address {127.0.0.1:9200}, bound_addresses {127.0.0.1:9200}
[2016-01-13 12:28:49,125][INFO ][node                     ] [MasterScrapper] started
[2016-01-13 12:28:49,316][INFO ][license.plugin.core      ] [MasterScrapper] license [414cb515-23b0-4a78-8af8-fcf060a1071f] - valid
[2016-01-13 12:28:49,317][ERROR][license.plugin.core      ] [MasterScrapper] 
#
# License will expire on [Sunday, February 07, 2016]. If you have a new license, please update it.
# Otherwise, please reach out to your support contact.
# 
# Commercial plugins operate with reduced functionality on license expiration:
# - marvel
#  - The agent will stop collecting cluster and indices metrics
[2016-01-13 12:28:49,351][INFO ][gateway                  ] [MasterScrapper] recovered [11] indices into cluster_state
[2016-01-13 12:28:55,106][INFO ][cluster.service          ] [MasterScrapper] added {{JG}{5OJu376pS7Gz4HSsZV-yFA}{127.0.0.1}{127.0.0.1:9301}{client=true, data=false, local=false, master=false},}, reason: zen-disco-join(join from node[{JG}{5OJu376pS7Gz4HSsZV-yFA}{127.0.0.1}{127.0.0.1:9301}{client=true, data=false, local=false, master=false}])
[2016-01-13 12:29:52,463][INFO ][cluster.service          ] [MasterScrapper] removed {{JG}{5OJu376pS7Gz4HSsZV-yFA}{127.0.0.1}{127.0.0.1:9301}{client=true, data=false, local=false, master=false},}, reason: zen-disco-node_failed({JG}{5OJu376pS7Gz4HSsZV-yFA}{127.0.0.1}{127.0.0.1:9301}{client=true, data=false, local=false, master=false}), reason transport disconnected

Cluster Health (GET _cluster/health)

{
  "cluster_name": "Scrapper",
  "status": "yellow",
  "timed_out": false,
  "number_of_nodes": 1,
  "number_of_data_nodes": 1,
  "active_primary_shards": 24,
  "active_shards": 24,
  "relocating_shards": 0,
  "initializing_shards": 0,
  "unassigned_shards": 24,
  "delayed_unassigned_shards": 0,
  "number_of_pending_tasks": 0,
  "number_of_in_flight_fetch": 0,
  "task_max_waiting_in_queue_millis": 0,
  "active_shards_percent_as_number": 50
}

Host:

localhost (running everything on my local machine.)

Elasticsearch version 2.1.1 (downloaded ran from the official website)
Java Clientlib version org.elasticsearch - elasticsearch - 2.1.1

Installed plugins in /Users/myusername/libs/elasticsearch-2.1.1/plugins:
    - license
    - marvel-agent

config/elasticsearch.yml (Exact configuration)

cluster.name: "Scrapper"
node.name: "MasterScrapper"
node.master: true
node.data: true
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["127.0.0.1"]
network.host: "127.0.0.1"
transport.tcp.port: 9300

OS

OsX El Captain 10.11.2

java

$ java -version
java version "1.8.0_66"
Java(TM) SE Runtime Environment (build 1.8.0_66-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode)

What happens if you clear the discovery.zen.ping.unicast.hosts property if the server? — Jonas Bergström, Jan 11 '16 at 09:30
Judging by your logs above, the Java client is started before (on Jan 8th) your ES master (on Jan 9th). Can you ensure that you start your master node and only when it's up you start your Java client? — Val, Jan 11 '16 at 12:03
In `discovery.zen.ping.unicast.hosts` values drop the transport port number, that is, set it to `["127.0.0.1"]` only. Does that help? — bittusarkar, Jan 12 '16 at 05:05
will update everything. and let you know. Sorry for late response, I was travelling. — Jigar, Jan 13 '16 at 06:35
But yes, I have tried removing the port, that was my first try, but the documentation says that one can add port too. http://tinyurl.com/jlnuztp — Jigar, Jan 13 '16 at 06:44
tried removing the port, and posted latest logs. As expected, didn't fix anything. — Jigar, Jan 13 '16 at 10:56
Can someone please replicate this on their machine, pretty please. — Jigar, Jan 13 '16 at 10:57

Jigar · Accepted Answer · 2016-01-21T08:37:54.077

5

There are multiple issues related to license plugin and ES 2.0

I made it work by removing the license plugin from ES master node. And the java client was able to connect. Yeh!

For more details visit: https://discuss.elastic.co/t/elasticsearch-2-1-1-on-localhost-java-client-error-masternotdiscoveredexception-waited-for-30s/39294

edited Jan 21 '16 at 08:37

answered Jan 15 '16 at 05:36

Jigar

8,762
6
25
26

score -1 · Answer 2 · answered Jan 13 '16 at 13:43

-1

It seems that your cluster doesn't have master elected, which means that there are some issues. Please look at your logs and see what you can find there. Do you have minimum_master_nodes property set in your ElasticSearch configuration ? If you do, check if the number of master eligible nodes in your cluster is sufficient for the master to be elected (at least the number specified by that property).

Try curl -XGET 'localhost:9200/_cluster/health?pretty'

answered Jan 13 '16 at 13:43

udit mittal

529
3
14

If you search this page for elect you will find that master is elected in elastic search logs. So, I wonder how it seems that there is no master elected. Hence the point regarding minimum_master_nodes or cluster health seems moot. Regardless, I shall add cluster health output to my question. Thanks. – Jigar Jan 14 '16 at 23:45

Elasticsearch 2.1.1 on localhost Java client error: MasterNotDiscoveredException[waited for [30s]]

Java Client Code

Java Client Logs:

Elasticsearch Logs:

Cluster Health (GET _cluster/health)

2 Answers2