google app script Exceeded memory limit

Question

May be this question already asked, but that won't solve my problem.

I try to save data's into google spreadsheet using google app script. But it shows Exceeded memory limit error.

following my code:

//new
function getNewTitle() {
    var url = "https://www.reddit.com/r/DigitalMarketing.rss?limit=100&after=0";
    var fromText = '</updated><title>';
    var toText = '</title>';
    var content = UrlFetchApp.fetch(url).getContentText();

    var scraped = Parser.data(content).from(fromText).to(toText).iterate();
    return scraped;
}

function getNewContent() {
    var url = "https://www.reddit.com/r/DigitalMarketing.rss?limit=10&after=0";
    var content = UrlFetchApp.fetch(url).getContentText();

    var document = XmlService.parse(content);
    var root = document.getRootElement();
    var atom = XmlService.getNamespace('http://www.w3.org/2005/Atom');
    Logger.log(atom);

    var fromText = '<content type="html">&lt;!-- SC_OFF --&gt;&lt;div class=&quot;md&quot;&gt;&lt;p&gt;';
    var toText = '</div>';


    var scraped = Parser.data(content).from(fromText).to(toText).iterate();

    return scraped;
}

function getNewLink() {
    var url = "https://www.reddit.com/r/DigitalMarketing.rss?limit=10&after=0";
    var fromText = '<link href="';
    var toText = '" /><updated>';
    var content = UrlFetchApp.fetch(url).getContentText();

    var scraped = Parser.data(content).from(fromText).to(toText).iterate();

    return scraped;
}

function SAVE_DATA() {
  var sheet  = SpreadsheetApp.openById('1No3m_FnhyxIaxj2zSlbHrg8HLBJULGQ2bda65hpKlyY').getSheetByName('sample'); 
 var content   = getNewContent();
  var title   = getNewTitle();
  var link   = getNewLink();
  Logger.log(title[1]);
  for(var i =0; i < title.length; i++) { 
    sheet.appendRow([ 'Reddit','wordpress', title[i], link[i], content[i]]);
  }
}
//new

In my above code am tried to save the data from url.

But i get Exceeded memory limit error.

In my Log i got this message

[18-07-21 05:33:29:719 PDT] [Namespace: prefix "" is mapped to URI "http://www.w3.org/2005/Atom"]

Please help me to fix this error...!

Thanks in advance.

Answer 1

I think that the reason of the error is that </div> of var toText = '</div>'; is not included in content retrieved from https://www.reddit.com/r/DigitalMarketing.rss?limit=10&after=0. So how about this modification?

Modification points :

• </div> of var toText = '</div>'; is not included in content. So in this modification, I used </content>. Because you are using '<content type="html">&lt;!-- SC_OFF --&gt;&lt;div class=&quot;md&quot;&gt;&lt;p&gt;' for fromText.
• setValues() instead of appendRow() is used for putting the values.
  • You can see the difference of the cost between setValues() and appendRow() at here.

Modified script :

1. For getNewContent()

Please modify from

From :

var toText = '</div>';

To :

var toText = '</content>';

2. For SAVE_DATA()

Please modify as follows.

function SAVE_DATA() {
  var sheet  = SpreadsheetApp.openById('1No3m_FnhyxIaxj2zSlbHrg8HLBJULGQ2bda65hpKlyY').getSheetByName('sample');
  var content = getNewContent();
  var title   = getNewTitle();
  var link   = getNewLink();
  var values = title.map(function(e, i){return [e, link[i], content[i]]});
  sheet.getRange(sheet.getLastRow() + 1, 1, values.length, values[0].length).setValues(values);
}

Note :

• In this modification, I used var toText = '</content>'; for getNewContent(). If you want to retrieve other range of the site, please modify this.
• About the URL, limit=100 for the title is set. But limit=10 is set for the link and content. So when the values are retrieved and put them to Spreadsheet, link and content become undefined from 11 row.
  • If you have already known this, please ignore this.

Reference :

• Easy data scraping with Google Apps Script in 5 minutes
  • Parser is a GAS library. You can check at here.

If I misunderstand your question, I'm sorry.