1

I have a google sheet with the following formula that works fine: =index(importxml("https://sullygnome.com/channel/xqcow/30","//div[@class='InfoStatPanelTLCell']"),1,1)

Trying to do it with google apps script but I get an error.

Using the following:

function parseXml() {
let url = "https://sullygnome.com/";
let data = UrlFetchApp.fetch(url).getContentText();
Logger.log(data);
}

I get this error:

Exception: Address unavailable: https://sullygnome.com/
parseXml    @ Code.gs:3

Tried using the same code with google.com instead and it worked:

function parseXml() {
let url = "https://google.com/";
let data = UrlFetchApp.fetch(url).getContentText();
Logger.log(data);
}

Any ideas how to tackle this? Thanks

Rubén
  • 34,714
  • 9
  • 70
  • 166
Maor U
  • 19
  • 2
  • Change your code to `let response = UrlFetchApp.fetch(url);` and then check the error code `response.getResponseCode();`. I'm guessing the website is blocking your request as it's not called from a web browser – Greg Sep 26 '21 at 11:13
  • @Greg changed to `function parseXml() { let url = "https://sullygnome.com/"; let response = UrlFetchApp.fetch(url); Logger.log(response.getResponseCode()); }` Getting the same error, anyway to bypass it? – Maor U Sep 26 '21 at 11:35

1 Answers1

0

The website rejects your request because it's doesn't come from a web browser.

However it looks like you can get around this issue by passing headers and setting the user agent to web browser (e.g Chrome).

For example:

function parseXml() {
    let url = "https://google.com/";

    var options = {
        headers: {
            'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36',
        }
    };

    let data = UrlFetchApp.fetch(url, options).getContentText();
    Logger.log(data);
}

Note: I've only tested running the requests with Postman. I simply ran a normal GET requests and then altering the user agent, to get the correct response. It maybe that you'll nee to play around with the code.

Greg
  • 4,468
  • 3
  • 16
  • 26
  • Thank you for that. Tried to do the workaround, and changed the user agent to different agents, but still no success. – Maor U Sep 29 '21 at 07:19
  • It looks like the request is being blocked by the third party website. Google App Script provide their IP ranges online, which can be used by the website to block requests. One way to get around this is to use a Web Proxy. However it looks like Google App Script doesn't support proxies. – Greg Sep 29 '21 at 18:55