4

I'm using the Request module to fetch html pages, it works great but after some time I get errors ENOTFOUND and EADDRINFO on some of the requests. After looking around online I found some similar issues (#699, #5488, #697) and what I understand from it is that I'm probably not consuming some (or all?) of the responses and that at some point the app reaches the socket pool max size.

I'm not really sure about all that, but I'm thinking about increasing globalAgent.maxSockets (to 100?) and always consuming the response by calling response.resume() at the beginning of the request callback. Does any of that make sense ? Or am I way off the tracks ?

I've looked into Node's code to try to understand what resume() does but I'm not sure I found the right function (#L698).

Thanks for your help.

jc-tzn
  • 93
  • 1
  • 11
  • can you describe the environment you're running this in? do you see the same symptoms in a local environment as you would on say, AWS, Heroku, etc? – aembke Sep 05 '14 at 21:07
  • I don't recall seeing that issue on the dev environment. The prod env is on AWS, with Node 0.10.21 – jc-tzn Sep 07 '14 at 12:40
  • @jc-tzn did you find a solution? I'm having similar issues. – Ace Aug 27 '15 at 14:43
  • I did what I said I'll try and that solved the issue (not really sure why though). So before initializing the server I set `http.globalAgent.maxSockets = 100` and `https.globalAgent.maxSockets = 100` and every time I use the `request` module I add this in the callback: `if (response) { response.resume(); }` – jc-tzn Aug 28 '15 at 08:16
  • @jc-tzn you should add your solution as an answer and choose it as correct so that people searching for a similar issue will know that it is solved in search results. – kanaka Jul 07 '16 at 22:42

1 Answers1

0

Per kanaka's suggestion I'm answering my own question with the solution I found:

I did what I said I'll try and that solved the issue (not really sure why though). So before initializing the server I set http.globalAgent.maxSockets = 100 and https.globalAgent.maxSockets = 100 and every time I use the request module I add this in the callback: if (response) { response.resume(); }

Maybe that will help someone else :)

jc-tzn
  • 93
  • 1
  • 11