EADDRNOTAVAIL on elasticsearch high load

Question

I am using es module to communicate with elasticsearch server.

When I do massive insert I got error after about 10000 queries:

Trace: { [Error: connect EADDRNOTAVAIL]
  code: 'EADDRNOTAVAIL',
  errno: 'EADDRNOTAVAIL',
  syscall: 'connect' }

Update:

OS X 10.9.3

❯ elasticsearch -v
Version: 1.2.0, Build: c82387f/2014-05-22T12:49:13Z, JVM: 1.7.0_45

❯ node -v
v0.10.26

Update 2

According to Alex advice:

❯ ulimit -a
-t: cpu time (seconds)              unlimited
-f: file size (blocks)              unlimited
-d: data seg size (kbytes)          unlimited
-s: stack size (kbytes)             8192
-c: core file size (blocks)         0
-v: address space (kbytes)          unlimited
-l: locked-in-memory size (kbytes)  unlimited
-u: processes                       709
-n: file descriptors                256

❯ sudo sysctl -w kern.maxfilesperproc=20000
Password:
kern.maxfilesperproc: 10240 -> 20000

But same result: after about 10000 operations it fails with the same errors.

Update 3

I run command netstat -an | grep -e tcp -e udp | wc -l during my script execution thich last about 30 seconds:

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     109

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     110

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     110

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     110

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     114

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     114

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     112

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     114

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     114

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     114

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     113

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     111

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     112

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     112

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     112

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     112

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     110

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     112

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     112

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     112

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     110

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     112

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     110

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     110

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     108

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     108

[user@mac] ~  
❯  netstat -an | grep -e tcp -e udp | wc -l
     108

[user@mac] ~  
❯

Can you add operating system and version, version of elasticsearch and version of node.js you are running? if linux can your run netstat -an | grep -e tcp -e udp | wc -l and post the output here? Preferably when you are running your insert. — John Petrone, Jun 01 '14 at 03:46
@eft can you run this command (works on a Mac too) and add the output to your question? Best if run during a massive insert, even better if you run it several times right up to and during the failure: netstat -an | grep -e tcp -e udp | wc -l — John Petrone, Jun 03 '14 at 19:41
@JohnPetrone result of script is nearly the same during massive elastic search insert: about `112` (Update 3) — Maxim Yefremov, Jun 05 '14 at 04:08
Honestly I'm a bit stumped - I'd recommend you contact the author of the package you are using or perhaps consider using some more widely used like https://github.com/elasticsearch/elasticsearch-js — John Petrone, Jun 05 '14 at 04:23

score 0 · Answer 1 · edited May 23 '17 at 12:15

0

I suppose node.js insert requests are processed in async mode, so it creates more connections, elasticsearch could handle. Could you use some delay or connection pool (not sure how it's done in node.js)?

update

use also ulimit -a to check max number of open handlers and find a way to increase it using this post Maximum number of open filehandles per process on OSX (and how to increase)

edited May 23 '17 at 12:15

Community

1
1

answered Jun 03 '14 at 12:03

Alex

1,210
8
15

No, Each next insertion goes only after preveous insertion to elastic search done – Maxim Yefremov Jun 03 '14 at 16:04
Ok, I tried to increase max number of open handlers but no result (Update 2) – Maxim Yefremov Jun 05 '14 at 04:00

score 0 · Answer 2 · answered Jun 06 '14 at 10:14

0

If these inserts are running from a script and some data source you have i wouldn't recommend doing an insert for each entry. Instead I would recommend you use the bulk uploading API. The node library you're using supports this and it is documented here.

answered Jun 06 '14 at 10:14

Garry Welding

3,599
1
29
46

EADDRNOTAVAIL on elasticsearch high load

Update:

Update 2

Update 3

2 Answers2