1

I just renew the example and also add the How I want the code to work

I'm quite new with map() function. I want to change the for and if condition to map() because it takes to long for processing a lot of data. Or you guys have any idea to make my code more faster and efficient to work, it can be really helpful.

How I want my code's work:
1. Find the row that have empty value on Column 3 from Table of Data
2. Concate or merge the value of Column 1 and Column 2 from Table of Data
3. Find the same value with the merged value in Table of Source_data
4. If the merged value is same with the value of Column 1 on Table of Source_data, then Get the data of column 2, Column 3, and Column 4
5. Write the data from Table of Source_data (Column 2, Column 3, Column 4) on the Column 3, Column 4, and Column 5 of Table of Data (Result like The Expected Output)

Thank you!

Table of Data:
Column 1 Column 2 Column 3 Column 4 Column 5
lose data data1 data2 data3
Second row
Second row
Second row data4 data5 data6

Table of Source_Data:

Column 1 Column 2 Column 3 Column 4
losedata data1 data2 data3
Secondrow data4 data5 data6

Table of Data: (Expected Output)

Column 1 Column 2 Column 3 Column 4 Column 5
lose data data1 data2 data3
Second row data4 data5 data6
Second row data4 data5 data6
Second row data4 data5 data6
function main3() {
  var ss = SpreadsheetApp.getActiveSpreadsheet();
  var sheet = ss.getSheetByName('Sheet1');
  var data = sheet.getDataRange().getValues();
  var source_file = SpreadsheetApp.openById("").getSheetByName("");
  var source_data = source_file.getRange(2, 1, source_file.getLastRow() - 1, source_file.getLastColumn()).getValues();

  var headerName1 = "Column 1";
  var headerName2 = "Column 2";
  var header = data.shift();
  var header_index1 = header.indexOf(headerName1);
  var header_index2 = header.indexOf(headerName2);

 
  // To find empty row with specific number of column
  for (var i = 1; i < data.length; i++) {
    if (data[i][2] == "") {

      // merge 2 column
      // column 1: lose and column 2: data
      // var concat will generate the merge of those two column -> losedata
      var concat = data.map((row, i) => ([data[i][header_index1] + data[i][header_index2]]));

      // find the same value with the value of merged columns
      // this will find the same data on source_data (Source Spreadsheet) like "losedata"
      var matching = concat.map(row => {
        var row_match = source_data.find(r => r[0] == row[0])
        return row_match ? [row_match[3], row_match[4], row_match[5]] : [null, null, null]
      });
// write the value to the table of Data
   sheet.getRange(2, 3, matching.length, matching[0].length).setValues(matching);
    }
  }

}
Asking Bob
  • 43
  • 1
  • 7
  • Could you provide sample `data` array using `console.log(JSON.stringify(data))`? – TheMaster Jul 27 '22 at 09:29
  • Maybe it is better to setup a [sample sheet](https://docs.google.com/forms/d/e/1FAIpQLSeprZS3Al0n7JiVQIEiCi_Ad9FRXbpgB7x1-Wq6iAfdmVbWiA/viewform) with input and expected output. Based on your script you write to the sheet with every itteration, this is time consuming. – RemcoE33 Jul 27 '22 at 10:05
  • Do note that [your email address can also be accessed by the public](https://meta.stackoverflow.com/questions/394304/), if you share Google files. – TheMaster Jul 27 '22 at 10:07
  • See https://stackoverflow.com/a/57836700 Call `setValues()` outside the loop – TheMaster Jul 27 '22 at 10:11
  • So, I want to write the data from Source_data table to the table of data that have a empty row on "Column 3" . I think i must using setValue() than setValues() inside the loop, right? – Asking Bob Jul 27 '22 at 10:31
  • The main problem is I have a lot of data and the execute process takes to long if I use the for loop with if condition to find the empty row and I will get a timeout for that. That is why i ask about to change the for loop and if condition into using Map() because i think it will run faster (maybe) – Asking Bob Jul 27 '22 at 10:39
  • You can also use vlookup formula to do this. – TheMaster Jul 27 '22 at 10:58
  • Just remove the loop:`for (var i = 1; i < data.length; i++) { if (data[i][2] == "") {` and closing `}}` – TheMaster Jul 27 '22 at 11:02
  • if i remove the loop and then how can i find the empty row? Because i want this script works only for the empty rows – Asking Bob Jul 27 '22 at 11:09
  • There is no significant speed difference between a loop `for()` and method `array.map()`. If you want to speed up the code you need to avoid of the many calls to the server with `setValues()`. The best practice is to process all the data as a 2d array on client side and set the 2d array on the sheet with one `setValues()`. – Yuri Khristich Jul 27 '22 at 15:38

2 Answers2

1

As far as I can tell the problem is in setValues() inside the loop. It doesn't matter if the loop is for() or map(). If you want to speed up the script you have to reduce calls to the server. I believe in this case you can process all data on client side as a 2d array and set it on the sheet all at once with just one setValues():

function myFunction() {
  // get the destination sheet and data
  var ss = SpreadsheetApp.getActiveSpreadsheet();
  var sheet = ss.getSheetByName('Sheet1');
  var [header, ...data] = sheet.getDataRange().getValues();

  // get the source data
  var source_file = SpreadsheetApp.openById('').getSheetByName('');
  var [_, ...src_data] = source_file.getDataRange().getValues();

  // make source object from the source data {col1:[col2, col3, col4], ...}
  var obj = {};
  src_data.forEach(x => obj[x[0]] = x.slice(1));

  // loop through all the data and add cells from the object
  // whenever the object has the key (key is col1 + col2 of current row)
  for (let row in data) {
    var key = data[row][0] + data[row][1];
    if (key in obj) data[row] = [data[row][0], data[row][1], ...obj[key]];
  }

  // make the table from the updated data and set the table on the sheet
  var table = [header, ...data];
  sheet.getDataRange().setValues(table);
}

Data (original):

enter image description here

Source:

enter image description here

Data (results):

enter image description here

Update

It turned out the actual data has another columns.

Data (original):

enter image description here

Source:

enter image description here

Data (results):

enter image description here

The updated code to copy the blue columns from source to the data sheet (based on 'keys' in yellow columns) is here:

function myFunction() {
  // get the destination sheet and data
  var ss = SpreadsheetApp.getActiveSpreadsheet();
  var sheet = ss.getSheetByName('Sheet1');
  var [header, ...data] = sheet.getDataRange().getValues();

  // get the source data
  var source_file = ss.getSheetByName('Source');
  // var source_file = SpreadsheetApp.openById('').getSheetByName('');
  var [_, ...src_data] = source_file.getDataRange().getValues();

  // make source object from the source data {col1:[col2, col3, col4], ...}
  var obj = {};
  src_data.forEach(x => obj[x[0]] = x.slice(3));

  var headerName1 = "Column 1";
  var headerName2 = "Column 5";
  var header_index1 = header.indexOf(headerName1);
  var header_index2 = header.indexOf(headerName2);

  // loop through all the data and add cells from the object
  // whenever the object has the key (key is col1 + col2 of current row)
  for (let row in data) {
    var key = data[row][header_index1] + data[row][header_index2];
    if (key in obj) data[row] = [...data[row].slice(0,5), ...obj[key]];
  }

  // make the table from the updated data and set the table on the sheet
  var table = [header, ...data];
  sheet.getDataRange().setValues(table);
}
Yuri Khristich
  • 13,448
  • 2
  • 8
  • 23
  • I'm sorry for my knowledge. Can you explain the part var [header, ...data] = sheet.getDataRange().getValues();? – Asking Bob Jul 27 '22 at 17:02
  • How can I get the header values? – Asking Bob Jul 27 '22 at 17:11
  • `var [header, ...data] = sheet.getDataRange().getValues();` this line gets you `header` (the first row) and `data` (the rest of the rows) see 'spread' syntax https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Spread_syntax – Yuri Khristich Jul 27 '22 at 17:36
  • I already testing the code, but there is an error that happen because of the numbers of column from table of data is not equal to the numbers of column from table of source_data – Asking Bob Jul 27 '22 at 20:40
  • I'm sure there either is some crucial difference between the tables in your question and your real sheets or you've done something wrong when pasted my code in your script. It works fine in my test sheet: https://docs.google.com/spreadsheets/d/1JVfMOlTZRIEFg2m1H_gKqTwMYDAFVL6BKh_Npxcxmbg/edit?usp=sharing I could try to figure it out if you share a sample your real sheets (without sensitive information of course). Probably you can just paste the samples of your data into my spreadsheet if you don't want post any links here – Yuri Khristich Jul 28 '22 at 07:19
  • Okay, i'll edit your spreadsheet and giving you my real case – Asking Bob Jul 28 '22 at 08:24
  • I've just edit your spreadsheet, thank you – Asking Bob Jul 28 '22 at 08:31
  • Well, the real data has the several additional columns in both tables. I fixed the code to handle the real data. Try it. – Yuri Khristich Jul 28 '22 at 08:57
0

A better way to look for empty cells is using the Range.getNextDataCell(). This will look for the first empty cell. Assuming there are no empty cells in the column it will find the last empty cell. This is the same as Ctrl+[arrow key]. You can also use it to find empty columns by using Direction.NEXT or PREVIOUS.

Actually it's not the empty cell it returns but the boundary cell containing data of the direction you are looking. So be sure and add 1 or subtract 1 if looking UP.

function getEmptyCell() {
  try {
    let spread = SpreadsheetApp.getActiveSpreadsheet();
    let sheet = spread.getSheetByName("Sheet3");
    let column = 1;
    // In these 2 examples cell A6 is empty then A16.  Column B has 20 values, more than A
    // This first example finds the first empty cell starting from the 1st row down
    let row = sheet.getRange(1,column).getNextDataCell(SpreadsheetApp.Direction.DOWN).getRow();
    console.log(row);
    // In this example the first empty cell starting from the last row up.
    row = sheet.getRange(sheet.getLastRow()+1,column).getNextDataCell(SpreadsheetApp.Direction.UP).getRow();
    console.log(row);
  }
  catch(err) {
    Logger.log("Error in getEmptyCell: "+err)
  }
}
6:51:01 AM  Notice  Execution started
6:51:03 AM  Info    5
6:51:03 AM  Info    15
6:51:03 AM  Notice  Execution completed
Rubén
  • 34,714
  • 9
  • 70
  • 166
TheWizEd
  • 7,517
  • 2
  • 11
  • 19