sending request through proxy. request library works, axios does not

Question

I am trying to update some old code to get rid of the request package since it is no longer maintained. I attempted to replace a proxy request with axios, but it doesn't work (I just get a timeout). Am I missing an axios config somewhere? The example using the request package works fine.

FAILS

export function sendAxiosApiRequest(enableProxy, proxyIndex,url,filepath?:string):object {
  //https://support.zyte.com/support/discussions/topics/22000014602
  let ca='READ IN FILE HERE'
  
    let getOptions = {
        url: url,
        httpsAgent: tunnel.httpsOverHttp({
          ca: ca,
          proxy: {
            host: 'http://MY_API_KEY:@proxy.crawlera.com',
            port: '8011',
          },
        }),
        proxy: false, //disable auto config, bc we set it manually
    } as any;
    console.log({getOptions})

    return new Promise(resolve => {
        try {
            axios.get(getOptions,(err,response,html)=>{
                if(err){
                    console.log(err);
                    resolve(false);
                }
                else {
                  try{
                    const output = JSON.parse(html);
                    resolve(output);
                  }catch(e){
                    console.log({html})
                    throw `ERROR parsing html: `+JSON.stringify(e)
                  }
                }
            })
        }
        catch (e) {
            console.log(`Err parsing result from sendApiRequest:`,e);
            resolve(false);
        }
    })
}

WORKS

export function sendRequestApiRequest(enableProxy, proxyIndex,url,filepath?:string):object {

  let ca='READ IN FILE HERE'
  let getOptions = {
      url: url,
      jar: true,
      followAllRedirects: false,
  } as any;
  //console.log({filepath})
      getOptions.proxy= 'http://MY_API_KEY:@proxy.crawlera.com'
      getOptions.ca=ca
          getOptions.requestCert =true
          getOptions.rejectUnauthorized= true

  return new Promise(resolve => {
      try {
          request.get(getOptions,(err,response,html)=>{
              if(err){
                  console.log(err);
                  resolve(false);
              }
              else {
                  const output = JSON.parse(html);
                  resolve(output);
              }
          })
      }
      catch (e) {
          console.log(e);
          resolve(false);
      }
  })
}

port as number? `port: 8011,`? – Juraj Jan 24 '23 at 10:32 — Juraj, Jan 24 '23 at 10:32

score 0 · Answer 1 · answered Jan 24 '23 at 17:04

0

Please have a look at the axios docs. The method signature for the get-requests is axios.get(url[, config]) but your first parameter is actually an object. You might want to use axios({}) and update your getOptions with the missing method key:

let getOptions = {
  url: url,
  method: 'get', // this was missing!
  httpsAgent: tunnel.httpsOverHttp({
    ca: ca,
    proxy: {
      host: 'MY_API_KEY:@proxy.crawlera.com', // http is not needed, but it was http but you use httpsAgent?!
      port: 8011,
    },
  }),
  proxy: false,
} as any;

answered Jan 24 '23 at 17:04

zangab

428
2
5

`get` is default. Are you sure there is no `axios.get(config)` even it is not in the doc? – Juraj Jan 24 '23 at 17:52
1

@Juraj yes, there isn't. you call `axios.get()` so the next param neesds to be an URL. I also tested it in a codepen, it does not work. `axios(options)` works though but u use `.get` in your code. you also wrap axios in a Promise which is unneccessary as axios is returning one, so you just can call `axios(options).then(response => { ... }).catch(error => { ... })`. Done. No need for any try/catch or `new Promise(...)`. – zangab Jan 24 '23 at 21:10
@zangab I tested adding `method:'get'` as a parameter. It did not work – Rilcon42 Jan 27 '23 at 19:44

sending request through proxy. request library works, axios does not

1 Answers1