55

I used Elasticsearch-1.1.0 to index tweets. The indexing process is okay. Then I upgraded the version. Now I use Elasticsearch-1.3.2, and I get this message randomly:

Exception happened: Error raised when there was an exception while talking to ES.
ConnectionError(HTTPConnectionPool(host='127.0.0.1', port=8001): Read timed out. (read timeout=10)) caused by: ReadTimeoutError(HTTPConnectionPool(host='127.0.0.1', port=8001): Read timed out. (read timeout=10)).

Snapshot of the randomness:

Happened --33s-- Happened --27s-- Happened --22s-- Happened --10s-- Happened --39s-- Happened --25s-- Happened --36s-- Happened --38s-- Happened --19s-- Happened --09s-- Happened --33s-- Happened --16s-- Happened 

--XXs-- = after XX seconds

Can someone point out on how to fix the Read timed out problem?

Thank you very much.

Aminah Nuraini
  • 18,120
  • 8
  • 90
  • 108
Hendra Bunyamin
  • 729
  • 1
  • 7
  • 11

6 Answers6

56

Its hard to give a direct answer since the error your seeing might be associated with the client you are using. However a solution might be one of the following:

1.Increase the default timeout Globally when you create the ES client by passing the timeout parameter. Example in Python 

es = Elasticsearch(timeout=30)

2.Set the timeout per request made by the client. Taken from Elasticsearch Python docs below.

# only wait for 1 second, regardless of the client's default
es.cluster.health(wait_for_status='yellow', request_timeout=1)

The above will give the cluster some extra time to respond

Skillachie
  • 3,176
  • 1
  • 25
  • 26
  • 1
    I have a similar issue and I increased the timeout to 60 seconds and still receive timeouts. I think the question is why is elastic search taking so long to respond? – digitaldavenyc Jan 31 '16 at 16:04
  • Depends on a variety of reasons. Do you have proper indices ? You might need to post a new question. – Skillachie Feb 01 '16 at 10:49
  • 5
    Is there a new question posted about the root cause? I'm seeing the same issue and I'm wondering what's the root cause of elasticsearch taking long to respond. – Suanmeiguo Sep 06 '16 at 20:47
48

Try this:

es = Elasticsearch(timeout=30, max_retries=10, retry_on_timeout=True)

It might won't fully avoid ReadTimeoutError, but it minimalize them.

Aminah Nuraini
  • 18,120
  • 8
  • 90
  • 108
6

Read timeouts can also happen when query size is large. For example, in my case of a pretty large ES index size (> 3M documents), doing a search for a query with 30 words took around 2 seconds, while doing a search for a query with 400 words took over 18 seconds. So for a sufficiently large query even timeout=30 won't save you. An easy solution is to crop the query to the size that can be answered below the timeout.

vlyubin
  • 586
  • 6
  • 18
5

For what it's worth, I found that this seems to be related to a broken index state.

It's very difficult to reliably recreate this issue, but I've seen it several times; operations run as normal except certain ones which periodically seem to hang ES (specifically refreshing an index it seems).

Deleting an index (curl -XDELETE http://localhost:9200/foo) and reindexing from scratch fixed this for me.

I recommend periodically clearing and reindexing if you see this behaviour.

Doug
  • 32,844
  • 38
  • 166
  • 222
2

Increasing various timeout options may immediately resolve issues, but does not address the root cause.

Provided the ElasticSearch service is available and the indexes are healthy, try increasing the the Java minimum and maximum heap sizes: see https://www.elastic.co/guide/en/elasticsearch/reference/current/jvm-options.html .

TL;DR Edit /etc/elasticsearch/jvm.options -Xms1g and -Xmx1g

Nomen Nescio
  • 347
  • 1
  • 7
2

You also should check if all fine with elastic. Some shard can be unavailable, here is nice doc about possible reasons of unavailable shard https://www.datadoghq.com/blog/elasticsearch-unassigned-shards/

Ryabchenko Alexander
  • 10,057
  • 7
  • 56
  • 88
  • +1 - we recently encountered this issue and the reason for timeout for us was unavailability of a shard due to low disk space on the machine https://stackoverflow.com/a/64117534/1526703 – Anupam Sep 29 '20 at 10:19