4

Hi I'm developing a small Node JS script for downloading to local files images from differents weather sat/radar sites. For exapmple https://radar.weather.gov/ridge/lite/KBYX_loop.gif when I open this url in my browser it opens perfectly the gif radar image. However when I call a get request using axios the response is 502 Bad Gateway. I'm working behind a company proxy so I have to pass my auth credentials. This code works perfectly with other sites for example for this site https://weather.cod.edu/satrad/assets/php/scripts/mkgif.php?parms=global-atlantic-14-1&start=445&end=445&rate=&pause=&checked=map&colorbar=data

Here is my code:

Axios.interceptors.request.use(
  function (config) {
    console.log(config);
    // console.log('Base URL: ', config.baseURL);
    // console.log('URL: ', config.url);
    // console.log('Params: ', config.params);
    return config;
  },
  function (error) {
    console.log(err);

    // Do something with request error
    return Promise.reject(error);
  }
);

Axios.interceptors.response.use(
  function (response) {
    console.log(response);
    return response;
  },
  function (error) {
    return Promise.reject(error);
  }
);

const download = async (fileUrl, outputLocationPath, config, params) => {
  const writer = fs.createWriteStream(outputLocationPath);
  return Axios({
    method: 'get',
    url: fileUrl,
    params: params,
    proxy: {
      host: config.host,
      port: config.port,
      auth: { username: config.user, password: config.pass },
    },
    responseType: 'stream',
  }).then((response) => {
    return new Promise((resolve, reject) => {
      response.data.pipe(writer);
      let error = null;
      writer.on('error', (err) => {
        error = err;
        writer.close();
        reject(err);
      });
      writer.on('close', () => {
        if (!error) {
          resolve(response);
        }
      });
    });
  })
  .catch(err =>  console.log(err));
};

And this is the response:

response: {
    status: 502,
    statusText: 'Bad Gateway',
    headers: {
      server: 'squid/4.13',
      'mime-version': '1.0',
      date: 'Wed, 30 Mar 2022 14:39:01 GMT',
      'content-type': 'text/html;charset=utf-8',
      'content-length': '3842',
      'x-squid-error': 'ERR_READ_ERROR 0',
      vary: 'Accept-Language',
      'content-language': 'en',
      'x-cache': 'MISS from ProxyB',
      'x-cache-lookup': 'MISS from ProxyB:3128',
      via: '1.1 ProxyB (squid/4.13)',
      connection: 'close'
    },
    config: {
      transitional: [Object],
      adapter: [Function: httpAdapter],
      transformRequest: [Array],
      transformResponse: [Array],
      timeout: 0,
      xsrfCookieName: 'XSRF-TOKEN',
      xsrfHeaderName: 'X-XSRF-TOKEN',
      maxContentLength: -1,
      maxBodyLength: -1,
      validateStatus: [Function: validateStatus],
      headers: [Object],
      method: 'get',
      url: 'https://cdn.star.nesdis.noaa.gov/GOES16/ABI/CONUS/11/latest.jpg',
      params: {},
      proxy: [Object],
      responseType: 'stream',
      data: undefined
    },

When I open the site in the browser I get the correct response and get the gif image.

I hope you can understand my issue and thanks in adv for any help or clarification.

Lazaro Falcon
  • 51
  • 1
  • 6
  • I have tried several sites and it seems a problem with http and https sites. However this https site works fine https://weather.cod.edu/satrad/assets/php/scripts/mkgif.php?parms=global-atlantic-14-1&start=445&end=445&rate=&pause=&checked=map&colorbar=data – Lazaro Falcon Mar 30 '22 at 15:26
  • I like to troubleshoot by verifying with less abstraction (in this case I choose `curl`). which worked for the noaa.gov URL in the code example above. – codeangler Mar 30 '22 at 15:36
  • I run >curl -x "http://user:pass@proxy:8080" --output file.gif "https://radar.weather.gov/ridge/lite/KBYX_loop.gif" on windows console and it works perfectly – Lazaro Falcon Mar 30 '22 at 18:43
  • So it has to bee some problem in the request with this site – Lazaro Falcon Mar 30 '22 at 18:50
  • 2
    Update: I have tested this in another network without this company proxy and works perfect. Anyone has an idea what could be the cause of this 502 Bad gateway response when call the get request behind my company proxy?? – Lazaro Falcon Mar 30 '22 at 22:20

0 Answers0