2

I'm trying to put together a function that will allow me to pull a column's info from one sheet to another based on a key column. This would work similar to an index match or vlookup in excel/google.

Sample Data:

enter image description here

What I've tried:

function vlookup(importFromSht, importToSht, importFromCompCol, importToCompCol,importFromCol, importToCol){
  var lastImportFromRN = importFromSht.getLastRow();
  var lastImportToRN = importToSht.getLastRow();
  var importFromCompArr =  importFromSht.getRange(2, importFromCompCol, lastImportFromRN, 1).getValues();
  var importToCompArr =  importToSht.getRange(2, importToCompCol, lastImportToRN, 1).getValues();
  var importFromArr =  importFromSht.getRange(2, importFromCol, lastImportFromRN, 1).getValues();
  var importToArr = [];

  for (var i in importToCompArr) {
    for (var j in importFromCompArr){
      if (importToCompArr[i].toString() == importFromCompArr[j].toString()) { 
        importToArr.push(importFromArr[j]);
      }
    }
  }

  //Paste to column
  importToSht.getRange(2,importToCol,importToArr.length,1).setValues(importToArr);
}

Parameters Defined

  • importFromSht - Sheet we are grabbing values from.
  • importToSht - Sheet values will be copied to.
  • importFromCompCol - Column (number) that has values to match on.
  • importToCompCol - Column (number) that has values to match on.
  • importFromCol - Column (number) that contains value that needs to be copied.
  • importToCol - Column (number) to copy value to.

Requirements:

  • There are over 6K rows minimum and could be many more thousand. Speed of execution is important. Is my approach the correct approach or are there more efficient methods? My script added about 30 seconds to my execution time.
  • Would like to call it as a function as I can see myself using this in other areas of this projects and other projects.
TheMaster
  • 45,448
  • 6
  • 62
  • 85
DanCue
  • 619
  • 1
  • 8
  • 17
  • 3
    `importToArr.length` might be be `0`. How is the speed important, when it doesn't work at all? – Martin Zeitler Feb 17 '20 at 04:01
  • @MartinZeitler Yes. That's what is happening but I'm not sure where I went wrong with my code. There are definitely matches. – DanCue Feb 17 '20 at 04:03
  • @MartinZeitler I added that as a requirement because I want to ensure that my approach is efficient, if not a recommendation would be appreciated. – DanCue Feb 17 '20 at 04:06
  • I just figured out why it did not work. A couple of issues. I was using i instead of j on the array it was being pushed to. Also, I had to use toString to convert the values to string when comparing the two values. I've edited my question. I'll leave it open in case there is a more efficient method. – DanCue Feb 17 '20 at 04:21
  • What does `Comp` mean? Explain your functions parameters in the question – TheMaster Feb 17 '20 at 04:27
  • @TheMaster good call. I'll do that. Comp means compare. – DanCue Feb 17 '20 at 04:28
  • 1
    A custom function can't be named `vlookup` because there is already a built-in function that has that name. Also bear in mind that the maximum execution time for custom functions is 30 seconds. Ref. https://developers.google.com/apps-script/guides/sheets/functions. Anyway if your script is working, the question is off-topic here but could be on-topic in [codereview.se]. – Rubén Feb 17 '20 at 05:39
  • 1
    @Rub It's not a custom function. He's trying to call it from another function – TheMaster Feb 17 '20 at 05:49
  • If you want efficient strategies, have you researched efficient strategies or algorithms? You're still expected to do your own research on execution times and available online information. Here's something to get you started: 1. https://stackoverflow.com/questions/500504/why-is-using-for-in-with-array-iteration-a-bad-idea 2. https://wikipedia.org/wiki/Binary_search_algorithm 3. https://stackoverflow.com/a/55467286/ 4. https://stackoverflow.com/a/56737759/ – TheMaster Feb 17 '20 at 05:55

2 Answers2

5

Issue:

  • Slow script:
    • O(n2): For every element in array 1, the array 2 is iterated from top to bottom. Even after finding a match in array 2, the loop is not broken(break), but the inner loop is completed until the end of the array2 unnecessarily.

    • getValues() is requested twice for two columns of the same sheet. Contact with spreadsheet is costly. So, limiting it is necessary.

Solution:

One possible solution to achieve O(n):

  • Create a new object using array 1 with key as 'the value to look for'. Then it's possible to directly access the value in this object each time for each value in array 2.

Sample script:

const ss = SpreadsheetApp.getActive();
/**
 * @param {GoogleAppsScript.Spreadsheet.Sheet} fromSht -Sheet to import from
 * @param {GoogleAppsScript.Spreadsheet.Sheet} toSht -Sheet to import to
 * @param {Number} fromCompCol -Column number of fromSht to compare
 * @param {Number} toCompCol -Column number of toSht to compare
 * @param {Number} fromCol -Column number of fromSht to get result
 * @param {Number} toCol -Column number of toSht to get result
 * @author https://stackoverflow.com/users/8404453
 */
function vlookup_custom(
  fromSht = ss.getSheetByName('Sheet1'),
  toSht = ss.getSheetByName('Sheet2'),
  fromCompCol = 1,
  toCompCol = 1,
  fromCol = 2,
  toCol = 2
) {
  const toShtLr = toSht.getLastRow();
  const toCompArr = toSht.getRange(2, toCompCol, toShtLr - 1, 1).getValues();
  const fromArr = fromSht.getDataRange().getValues();
  fromCompCol--;
  fromCol--;

  /*Create a hash object of fromSheet*/
  const obj1 = fromArr.reduce((obj, row) => {
    let el = row[fromCompCol];
    el in obj ? null : (obj[el] = row[fromCol]);
    return obj;
  }, {});

  //Paste to column
  toSht
    .getRange(2, toCol, toShtLr - 1, 1)
    .setValues(toCompArr.map(row => (row[0] in obj1 ? [obj1[row[0]]] : [null])));
}

Performance:

  • ~5s for 10000 rows in sheet1 and 10000 rows in sheet 2

References:

TheMaster
  • 45,448
  • 6
  • 62
  • 85
  • That shaved 30 seconds off. Excellent! – DanCue Feb 17 '20 at 16:56
  • Correct me if I am wrong but doesn't this grab more than the columns I need? Is the grabbing of values twice worse than grabbing all of that data and pasting it later? – DanCue Feb 17 '20 at 16:58
  • 1
    Reducing the data size is good, but not at the cost of two getValues() calls. One call with big data is better than two calls with smaller data. Test and see. Use `console.time()` – TheMaster Feb 17 '20 at 17:46
1

Unfortunately I am afraid there is no built-in function for this in Apps Script.

However, I have tried your custom function with 100 values and it took <3 seconds to run. I also ran it with 1000 values and my running time was around 40 seconds. It is not ideal but it works consistently. Here is the code I have used:

    var sheet = SpreadsheetApp.getActiveSpreadsheet();
    var importFromSht = SpreadsheetApp.getActive().getSheetByName('Sheet1'); //sheet we are grabbing the values of
    var importToSht = SpreadsheetApp.getActive().getSheetByName('Sheet2');  //sheet we are pasting our values
    var importFromCompCol = 2; // Column (number) that has values to match on.
    var importToCompCol = 2; // Column (number) that has values to match on.
    var importFromCol = 1; // Column (number) that contains value that needs to be copied.
    var importToCol = 1; // Column (number) to copy value to.
    
    function customVlookup (){
      var lastImportFromRN = importFromSht.getLastRow();
      var lastImportToRN = importToSht.getLastRow();
      var importFromCompArr =  importFromSht.getRange(1, importFromCompCol, lastImportFromRN, 1).getValues();
      var importToCompArr =  importToSht.getRange(1, importToCompCol, lastImportToRN, 1).getValues();
      var importFromArr =  importFromSht.getRange(1, importFromCol, lastImportFromRN, 1).getValues();
      var importToArr = [];
    
      for (var i in importToCompArr) {
        for (var j in importFromCompArr){
          if (importToCompArr[i].toString() == importFromCompArr[j].toString()) { 
            importToArr.push(importFromArr[j]);
          }
        }
      }
      //Paste to column
      importToSht.getRange(1,importToCol,importToArr.length,1).setValues(importToArr);
    }

A different and more efficient approach is to use IMPORTRANGE(URL of first sheet) to an intermediate sheet swapping the columns of the first sheet to then do a VLOOKUP on your second sheet. This would be way more efficient than doing it in Apps Script as it does not run in the memory and you avoid the issue of exceeding the execution time of a custom function.

In case you want to keep it as an Apps Script custom function Here are some suggestions on the documentation on how to improve your function’s efficiency.

Mateo Randwolf
  • 2,823
  • 1
  • 6
  • 17