Greetings overflowers,
I have create/update operations that require the writes to be synced to all replicas and their indexes to be refreshed, so I am using consistency: 'all' and refresh: true options. However, this is causing NodeJS elasticsearch client to always timeout (after 30000ms). Does it take that long? What is wrong?
Regards
You are asking a lot from Elastisearch. How many replicas do you have in your cluster?
If you have lot of replicas then having write consistency to all will take some time. Even if data integrity of utmost importance, I feel default value of quorum is enough.
Excellent source to understand write consistency
Last para from the above link
If your primary data-store is not elasticsearch it is almost certain that the all write-level is overkill. The performance impact will likely not be worth the extra guarantees of durability. As a final point, it should be noted that during a split-brain failure, where a single cluster has divided into two due to either a network partition or overloading, write-consistency guarantees will be rendered moot for obvious reasons.
And because of that I dont think having consistency level to all will save you from disaster.
setting refresh to true is overkill as you will be opening and writing so many segments every millisecond. ES is almsot real time as it refreshes every shard automatically every second.
Have a look at Elasticsearch document of Near Real Time Doc.
You will understand a lot from that.
If still you want to persist with those settings, then you need to increase timeout considerably from default value of 30 seconds.
