2

Overview: the workbook calculates an Overall Score for each Record (or row) based on the Category Scores for each Record. Category Scores are derived from computations performed on hundreds of data points for each Record. I compiled code to send an email onChange called checkComplete (below) based on several examples and tutorials I found online. checkComplete would only send an email if the status of the Record changed to "Complete." An onChange trigger is required since the workbook uses Importrange to load the raw data, and no human interaction takes place for an onEdit trigger.

The problem began when I noticed an email would be sent upon any change being made, not just the status change in column O from a blank cell to "Complete". But the code worked well enough until I made significant edits to the Google Sheets workbook that included:

  1. Modifying computational formulas to display a set number of decimals

    e.g. =if(round('2-Comp'!F14,4)=0,"",round('2-Comp'!F14,4))

  2. Moving columns around

    e.g. switching columns N and O, where N was the original column checked by the onChange trigger

  3. Updating source data, which therefore updated scores.

    e.g. a Record's data point changed from $52,420 to $54,323

Then the email bombardment began. The net result is that the script began to send an email for every Record starting with the first record, and would generate the following error messages in succession:

Service invoked too many times for one day: email. (line 47, file "checkComplete")

Exceeded maximum execution time

Service using too much computer time for one day

I have since commented out MailApp.sendEmail() to cease blowing up my inbox, but the script continues to generate the "Exceeded maximum execution time" and "Service using too much computer time for one day". My thought was that the script was working off all the changes described above, but now that this problem has persisted for over a week I decided to seek out help.

I thought the issue is with the array, specifically var oldValues, so I tried:

  1. to call flush() function to clear out the array, which did not solve the issue.

  2. to empty the array as per this thread - How do I empty an array in JavaScript?, but that did not resolve the issue either.

I now think the problem may be related to the logger, but to be honest I am stumped.

Below is the script's source code in its entirety:

checkComplete.gs

var admin_email='xxxxxxx@xxxxxxx.com'; //<- list of email recipients goes here

function checkComplete() {
  var sh = SpreadsheetApp.getActiveSheet();
  var values = sh.getRange('O14:O').getValues().join('-');
  if(PropertiesService.getScriptProperties().getKeys().length==0){ // first time you run the script
  PropertiesService.getScriptProperties().setProperty('oldValues', values);
  return;
}
  var oldValues = PropertiesService.getScriptProperties().getProperty('oldValues').split('-');
  var valuesArray = values.split('-');
  while (valuesArray.length>oldValues.length){
    oldValues.push('x'); // if you append some rows since last exec
  }
  Logger.log('oldValues = '+oldValues)
  Logger.log('current values = '+valuesArray)
  for(var n=0;n<valuesArray.length;n++){
    if(oldValues[n] != valuesArray[n]){ // check for any difference
      sendMail(n+1,valuesArray[n]);
    }
  }
   PropertiesService.getScriptProperties().setProperty('oldValues', values);

}
function sendMail(row,val){
  Logger.log('value changed on row '+row+' value = '+val+' ,  mail sent');
  var rank = SpreadsheetApp.getActiveSheet().getRange(row,1).getValue();
// other var that I deleted here to shorten this post
// MailApp.sendEmail(admin_email,'message that calls vars');

}

I expect to receive an email only upon the change of a Record's status, indicated in column O, from a blank cell to "Complete". All other data updates should not trigger any email. Any advice/help is appreciated. Thank you for taking the time to review my problem.

  • GMT, did you figure this out? Copying your code worked for me with no problems. – jdavis Sep 04 '19 at 12:45
  • Can I see your spreadsheet? – ziganotschka Sep 04 '19 at 13:39
  • @JonathanDavis, that is encouraging. No, the error log continues to grow each day. I wonder what the issue is. I am going to post a link to view the spreadsheet in a reply to zianotschka in case you were interested in taking a look. – GMT Skunkworks Sep 05 '19 at 14:07
  • @ziganotschka, sure, I just opened the spreadsheet up for viewing here https://docs.google.com/spreadsheets/d/1ydcXQilx6hxhI6HPpWTPT7Bq9a-gXPa7h8UC6KL9W8c/edit?usp=sharing. FYI, I continue to receive an error log that contains the following two message: "Exceeded maximum execution time" and "Service using too much computer time for one day" while MailApp.sendEmail() is commented out. – GMT Skunkworks Sep 05 '19 at 14:12
  • @GMTSkunkworks did you set up an installable trigger? I'm assuming so but if so, did you use onChange or onEdit? I'd check using the menu>current project triggers to ensure that you haven't added a bunch of triggers for the same function (which could explain the service invoked too many times for one day error). If so, you can use a script to remove them all quickly. I'll provide the code if that's the issue – jdavis Sep 05 '19 at 14:23
  • @GMT Skunkworks: I cannot work with your file and see the attached triggers due to your protection settings, but be aware that an onEdit trigger is not attached to a specific sheet - a change in any of the many other sheets you have in the spreadsheet will also fire the trigger. This means that the script will compare the values of the column O in the last edited sheet against the ones of the pre-last edited sheet. Chances are that all values will be completely different and each row will call `sendMail(row,val)`. Add an if statement `if(sh.getName()=="Dashboard){//continue the script...}` – ziganotschka Sep 05 '19 at 14:55
  • Also, when a new data row gets inserted //appended into "Dashboard" - will this row be inserted at the beginning or end of the sheet? – ziganotschka Sep 05 '19 at 14:56
  • @ziganotschka, I just opened the spreadsheet up so that you could edit it. Ah ha! What you say makes sense re: column O in any sheet. Where should I add the if statement you mention? Prior to the line 'var sh = SpreadsheetApp.getActiveSheet();'? Also, to answer your second question, the new row is added at the end of the sheet, into what should be the first empty row. That somewhat depends on our data entry personnel inputting the data into the correct row of the input sheet, which is a separate workbook. I noticed a few rows were blank, and corrected it in the data source workbook – GMT Skunkworks Sep 05 '19 at 15:22
  • @JonathanDavis, I just opened up the spreadsheet for editing so you could view the triggers. What you also say makes sense, but I believe that only one instance of the trigger is installed. What ziganotschka suggested as the error also makes sense, that the trigger is activating because of changes in any of the sheets. – GMT Skunkworks Sep 05 '19 at 15:24

1 Answers1

0

UPDATE

After specifying that the entries in Column O will be made by a human and will fire an onEdit() trigger, here is an easy and elegant solution making use of event objects:

 onEdit(e){
  if( e.range.getSheet().getName()=='Sheet1' &&
    e.range.getColumn()==15&&e.value=="Complete"&&e.oldValue!="Complete"){
    sendMail(e.range.getRow(),e.value);
   }  
}  

You can replace function checkComplete() and its onChange trigger by this onEdit function.


After reviewing your spreadsheet I noticed following issues:

  1. As mentioned in my comment, onEdit will be triggered by a change in any of the sheets within the spreadsheet, but you want the code to run only for changes in the sheet "Dashboard". And also, you do not want to compare the column O in Dashboard against the columns O in the other sheets - where all entries will be different and would trigger the call of sendMail. You should implement a condition statement to verify at the very beginning of "Dashboard that you are in the right sheet, before continuing with the rest of the code.

  2. Your function sendMail contains

  var rank = SpreadsheetApp.getActiveSheet().getRange(row,1).getValue();
  var totalranked = SpreadsheetApp.getActiveSheet().getRange(4,1).getValue();
  var address = SpreadsheetApp.getActiveSheet().getRange(row,2).getValue();
  var score = SpreadsheetApp.getActiveSheet().getRange(row,6).getValue();
  var outlook = SpreadsheetApp.getActiveSheet().getRange(row,14).getValue();
  var ildemand = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('1-Dmnd').getRange(row+1,9).getValue();
  var aldemand = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('1-Dmnd').getRange(row+1,14).getValue();
  var mcdemand = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('1-Dmnd').getRange(row+1,19).getValue();
  var increase = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('2-Comp').getRange(row,11).getValue();
  var occupancy = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('2-Comp').getRange(row,13).getValue();
  var medHHI = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('4-Demo').getRange(row+2,17).getValue();
  var medHV = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('4-Demo').getRange(row+2,19).getValue();
  var walkscore = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('3-Muni').getRange(row,19).getValue();
  var dmnd = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('1-Dmnd').getRange(row+1,5).getValue()
  var comp = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('2-Comp').getRange(row,5).getValue();
  var muni = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('3-Muni').getRange(row,5).getValue();
  var grade = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('3-Muni').getRange(row,17).getValue();
  var demo  = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('4-Demo').getRange(row+2,5).getValue();

This is 18 calls to SpreadsheetApp for each call of sendMail. And this multiplied by the amount of time sendMail will be called if you compare columns O of two different sheets against each other (see 1.)! Be aware that any requests to external services are very time consuming and inefficient and should be avoided, see Best Practices. You should retrieve all those variables only once, outside of sendMail. And given that after this the code of sendMail will be short, you can implement it directly within your for loop.

  1. If a row is appended in the beginning rather than the end of the sheet - this will shift the new values against the old values and thus, each row will trigger the call of sendMail. You can avoid but checking either the new contents are contained within the array with the old contents with indexOf.

  2. Go from the Apps Script editor UI to Edit->Current project's triggers and check if you accidentally attached multiple triggers to the file, which all run simultaneously.

Taking in consideration the points 1., 2. and 3., here is an improvement suggestion for your code:


var admin_email=XXXXX; //<- list email addresses here

function checkComplete() {
  var ss=SpreadsheetApp.getActive();
  var sh = ss.getActiveSheet();
  if(sh.getName()=="Dashboard"){
   var values = sh.getRange('O14:O').getValues().join('-');
   if(PropertiesService.getScriptProperties().getKeys().length==0){ // first time you run the script
    PropertiesService.getScriptProperties().setProperty('oldValues', values);
    return;
   }
   var oldValues = PropertiesService.getScriptProperties().getProperty('oldValues').split('-');
   var valuesArray = values.split('-');
   while (valuesArray.length>oldValues.length){
    oldValues.push('x'); // if you append some rows since last exec
   }
   Logger.log('oldValues = '+oldValues)
   Logger.log('current values = '+valuesArray)
   for(var n=0;n<valuesArray.length;n++){
    if(oldValues.indexOf(valuesArray[n])==-1){ // check for any difference
        Logger.log('value changed on row '+n+1+' value = '+valuesArray[n]+' ,  mail sent');
      sendMail(n+1,valuesArray[n]);
    }
   }  
   PropertiesService.getScriptProperties().setProperty('oldValues', values);
  }   
}


function sendMail(row,val){

   Logger.log('value changed on row '+row+' value = '+val+' ,  mail sent');

   var rank = sh.getRange(row,1).getValue();
   var totalranked = sh.getRange(4,1).getValue();
   var address = sh.getRange(row,2).getValue();
   var score = sh.getRange(row,6).getValue();
   var outlook = sh.getRange(row,14).getValue();
   var ildemand = ss.getSheetByName('1-Dmnd').getRange(row+1,9).getValue();
   var aldemand = ss.getSheetByName('1-Dmnd').getRange(row+1,14).getValue();
   var mcdemand = ss.getSheetByName('1-Dmnd').getRange(row+1,19).getValue();
   var increase = ss.getSheetByName('2-Comp').getRange(row,11).getValue();
   var occupancy = ss.getSheetByName('2-Comp').getRange(row,13).getValue();
   var medHHI = ss.getSheetByName('4-Demo').getRange(row+2,17).getValue();
   var medHV = ss.getSheetByName('4-Demo').getRange(row+2,19).getValue();
   var walkscore = ss.getSheetByName('3-Muni').getRange(row,19).getValue();
   var dmnd = ss.getSheetByName('1-Dmnd').getRange(row+1,5).getValue()
   var comp = ss.getSheetByName('2-Comp').getRange(row,5).getValue();
   var muni = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('3-Muni').getRange(row,5).getValue();
   var grade = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('3-Muni').getRange(row,17).getValue();
   var demo  = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('4-Demo').getRange(row+2,5).getValue();


      // MailApp.sendEmail(admin_email,'EARLY results for '+address+' are ready!','EARLY says the Site is '+outlook+' and ranks # '+rank+' out of '+totalranked+' with an overall Score of '+score+'. The projected Net Demand is '+dmnd+' with: IL '+ildemand+' units, AL '+aldemand+' units and MC '+mcdemand+' units. The competitive environment is '+comp+' with an occupancy rate of '+occupancy*100+'% for nearby properties, and an expected supply increase of '+increase*100+'%. The location is '+walkscore+', and its public schools have a grade of '+grade+' making it '+muni+'. The demographic trends are '+demo+' with Median HHI of $'+medHHI+' and a median Home Value of $'+medHV+'. For more information, please visit https://docs.google.com/spreadsheets/d/1ydcXQilx6hxhI6HPpWTPT7Bq9a-gXPa7h8UC6KL9W8c/edit?usp=sharing.');
    // add senior growth rate(s) to the Demographic summary sentence - Sept 3 2019
}

ziganotschka
  • 25,866
  • 2
  • 16
  • 33
  • 1
    Thank you for your detailed response, and suggested edit to the code. As for your points: 1. Understood and agreed; 2. Understood, I will review Best Practices and see if I can figure out how to implement the solution you suggest; 3. New data will always be appended to the next available blank row, never to the beginning of the sheet; and, 4. I can confirm that the Current Project's Triggers is "Showing 1 Trigger". I uncommented MailApp.sendEmail() so that once Google resets a new record will be added to the source workbook to see if the issue is resolved. Stay tuned... – GMT Skunkworks Sep 05 '19 at 20:13
  • Just run the code once with MailApp.sendEmail() and all variables calls within sendMail commented out,given that during the first run before the script properties get populated for the first time, the condition of ` if(oldValues.indexOf(valuesArray[n])==-1)` will always be fulfilled – ziganotschka Sep 06 '19 at 06:51
  • Thanks! Before I saw your msg, yesterday @ 4:22pm I ran the script, and @ 4:23pm to change the status of cells O71 and O72 on the Dashboard sheet to blank then to "Complete". The script's execution completed, but when I didn't receive any email I assumed that my account already had the error "Service using too much computer time for one day". That is when I replied to your message. I followed your instructions to comment out the items you listed, and run the script. Then I uncommented those items, and tested changing cell O472. The script's execution completed, but no email was received. – GMT Skunkworks Sep 06 '19 at 14:13
  • What is your intention? To send the emails from all the rows already in the sheet or only for new rows? Insert a new row after uncommenting to trigger a new email. – ziganotschka Sep 06 '19 at 14:35
  • Good question. Thinking in terms of new vs. existing rows, the answer would be when a new row is marked "Complete." Old rows should now all be "Complete," therefore, an email should have been sent already. A row populates from data imported to the 0-Inputs sheet. As our data entry team inputs values into the source Inputs workbook, the 0-Inputs sheet begins to populate with data cell by cell. We separated the two workbooks for security reasons. Thus, an email should be sent only once a new row on the Dashboard sheet is marked "Complete." – GMT Skunkworks Sep 06 '19 at 14:53
  • I should have stated more clearly that "A new row populates from data imported..." instead of saying that "A row populates..." – GMT Skunkworks Sep 06 '19 at 15:23
  • A human marks the row as complete once all data has been entered in the source workbook. – GMT Skunkworks Sep 09 '19 at 13:48
  • Then I suggest you the following solution: modify your trigger from "onChange" to "onEdit", so it will fire only when a human modifies the sheet. With the event objects e.value and e.oldValue you can verify either the value in column O has been set for the first time to Complete and if so - send an email. I will add this solution to my answer. – ziganotschka Sep 09 '19 at 14:55
  • Understood. However, the edit of a row's status to complete occurs in the source workbook, not the EARLY 2.0 (WIP) workbook where this script resides. A row's being marked complete in the EARLY 2.0 (WIP) workbook is via an Importrange() to the source workbook. Hence I thought onChange was correct. If you still think otherwise, I will change the trigger to onEdit. If I remember correctly, onEdit failed to activate the trigger when I tested the original script, but onChange successfully activated the trigger. – GMT Skunkworks Sep 09 '19 at 16:13
  • @GMT Skunkworks. The difference between onChange and onEdit is that onEdit allows you to use e.value and e.oldValue https://developers.google.com/apps-script/guides/triggers/events. It is true that onEdit does not get triggered by an update of importrange, but it does get triggered through human edit. So, if the marking as complete could be done in the final file - this would work. Otherwise, you need to stick to my initial solution and send new emails as soon as new data is added to the final spreadsheet via Importrange().You need to use for values in column O a unique ID identifying each row – ziganotschka Sep 12 '19 at 16:52
  • again, thank you for your time helping me resolve this issue. You have been terrific! A test just went through this morning, row 92 of the source data entry workbook's Dashboard sheet - https://docs.google.com/spreadsheets/d/1Lkd-zQlBpJab7TDZdd8qmMAnjalWt1itmitk4W_tQ_0/edit?usp=sharing ; however, no email triggered to alert us that the record in the workbook EARLY 2.0 (WIP) was complete. To finalize this, I assume that I should revert back to your initial solution. Please confirm and I will do so and run another test. – GMT Skunkworks Sep 13 '19 at 13:02
  • If there is no manual edit in your final sheet - then yes, you need to go back to the onChange() solution. But if your rows will be imported only when they are complete and they will be appended below the already existent - you do not need to loop through all rows each time. Just save during each script run the last Row in Script properties as OldLastRow and send the email only for the new rows (where row>oldLastRow); – ziganotschka Sep 13 '19 at 13:37