5

I am using request-promise module to check whether site is working with proxy or not. I am trying to find the proxies which is fast enough to answer in 5 seconds. Therefore I only add object if request doesn't timeout in 5 seconds.

For some proxies, even though promise resolves, node script hangs for a while. I can't find the reason of this delay. I see that it prints Done but it hangs. After 1 minute 10 seconds later, script exits. Is this hang due to my code, or operating system issue for open sockets etc?

const rp =  require('request-promise');
const testProxies = [
    {
       "ipAddress": "80.241.219.83",
       "port": 3128,
    },
    {
        "ipAddress": "45.55.27.246",
        "port": 80
    },
    {
        "ipAddress": "144.217.197.71",
        "port": 8080,
    },
    {
        "ipAddress": "104.131.168.255",
        "port": 80,
    },
    ];

function CheckSites(sitesArray,site) {
    let ps = [];
    for (let i = 0; i < sitesArray.length; i++) {
        let proxy = sitesArray[i];

        let ops = {
            method: 'GET',
            resolveWithFullResponse: true,
            proxy: 'http://' + proxy.ipAddress + ':' + proxy.port,
            uri:site,
        };
        let resp =  rp.get(ops);
        ps.push(resp);
    }
    return Promise.all(ps.map(function (p) {
        p.time =  Date.now();
        return p
            .then(function (a) {
                return {'header':a.headers,'time':Date.now() - p.time};
            })
            .timeout(5000)
            .catch(function (e) {
                return {};
            })
    }))
}
CheckSites(testProxies,'https://www.example.com').then(function (object) {
    console.log('Done!');
    console.log(object);
}).catch(function (err) {
    console.log('Exception: ' + err);
});
Meanteacher
  • 2,031
  • 3
  • 17
  • 48

1 Answers1

1

For your use case I suggest you to use the Promise.race() it behaves as Promise.all but you get the callback as soon as the fastest proxy responds.

I've investigated more on the bug, and it seems to be a request module issue, when you use timeout, they just don't close the connection and it's in hang-up state

Alexandru Olaru
  • 6,842
  • 6
  • 27
  • 53
  • I am getting `Done! `on console. It means all promises are resolved. Also even with `Promise.race()` it still hangs for a while then script exits – Meanteacher Aug 25 '17 at 08:22
  • @Meanteacher I updated my answer seems to be request module bug, on timeout https://github.com/request/request/issues/1676 they do not close the connection. – Alexandru Olaru Aug 25 '17 at 09:08
  • I checked your link which links to another issue. However, that user says it was an error on his code. I still can't understand the reason. Thanks anyway. – Meanteacher Aug 25 '17 at 09:23