1

I have multiple spreadsheets in a folder, for each spreadsheet and I would like to get the matching data from an external file. For example if my spreadsheet is called 'iphone 7' I would like to get in that sheet all the records from my external file containing 'iphone 7'

Here is how far I got (I am pretty new to scripting !) :

function myfunction()
{ 
 var root = DriveApp.getFoldersByName("Produits");                                        
  while (root.hasNext())
  { 
   var folder = root.next();    //If the folder is available, get files in the folder
   var files = folder.getFiles();                                                                       
   while(files.hasNext())       //For each file,                                                                    
   { 
    var spreadsheet = SpreadsheetApp.open(files.next()); 

    //import data from URL
     var csvUrl = "https://incensy.tempurl.host/test-ct-flux.csv";
  var csvContent = UrlFetchApp.fetch(csvUrl).getContentText();
  var csvData = Utilities.parseCsv(csvContent);
 
  var sheets = spreadsheet.getSheets()
  var sheetIndex=0
  var sheet = sheets[sheetIndex]
   sheet.getRange(2, 1, csvData.length, csvData[0].length).setValues(csvData);

//Only keep data that contains the file name
  var name = spreadsheet.getName();
  let range = sheet.getDataRange(),
      maxRows = sheet.getMaxRows(),
      srchCol_1 = 2,
      srchPatt_1 = new RegExp(name, "i"),
      newRangeVals = range.getValues().filter(r => r[0] && srchPatt_1.exec(r[srchCol_1])),   
      numRows = newRangeVals.length;  
  range.clearContent();
  sheet.getRange(2,1, numRows, newRangeVals[0].length).setValues(newRangeVals);
  console.log('myfunction')
  sheet.deleteRows(numRows + 1, maxRows - numRows);

   }
  }
 }

There is something wrong in the second part of the code I cannot figure out.

Antoine
  • 63
  • 6
  • I have to apologize for my poor English skill. Unfortunately, I cannot understand `For example if my spreadsheet is called 'iphone 7' I would like to get in that sheet all the records from my external file containing 'iphone 7'`. Can I ask you about the detail of your question? – Tanaike May 05 '22 at 06:12
  • Sure @Tanaike , I have multiple spreadsheets in a folder and I have an external CSV file. I would like to put the data from the external file in each spreadsheet, but only keep the data that contains the spreadsheet's name. – Antoine May 05 '22 at 06:17
  • Thank you for replying. About `I have an external CSV file`, where can you retrieve it? From your showing script, you want to retrieve the CSV data from the directlink of CSV data? – Tanaike May 05 '22 at 06:26
  • @Tanaike , yes I want to retrieve it fom an URL, I have created a sample file for testing purpose that I added in my code. My script will run on a schedule to retrieve data every day – Antoine May 05 '22 at 06:28
  • Thank you for replying. About `For example if my spreadsheet is called 'iphone 7' I would like to get in that sheet all the records from my external file containing 'iphone 7'`, in this case, you want to search the value from only `product_name`? And, you want to parse the CSV data? Or, you want to put each row is one cell of each row? – Tanaike May 05 '22 at 06:31
  • @Tanaike , not sure I understand "Or, you want to put each row is one cell of each row?" but for each row in CSV data that matches the spreadsheet's name I would like to get the matching row parsed in my spreadsheet. In this way to code seems to work when I use it on a single spreadsheet, the problem is when I try to loop into all the spreadsheets in the folder – Antoine May 05 '22 at 06:35
  • Thank you for replying. I deeply apologize for my poor English skill. I have one more question. When the Spreadsheet has already had the value, how do you want to do? You want to clear the sheet and put new data? Or, you want to append the data to the existing data? – Tanaike May 05 '22 at 06:38
  • No problem @Tanaike , thank you for taking the time to help. I would like the data to be cleared and replaced by the new data everyday. – Antoine May 05 '22 at 06:40
  • Thank you for replying. From your replying, I proposed a modified script as an answer. Could you please confirm it? If I misunderstood your question and that was not useful, I apologize. – Tanaike May 05 '22 at 06:45

1 Answers1

1

In your situation, how about the following modification?

Modified script:

function myfunction() {
  var keywords = ["sample1", "sample2"]; // Please set the keywords you want to filter to the column "C".

  // Retrieve CSV data.
  var csvUrl = "https://incensy.tempurl.host/test-ct-flux.csv";
  var csvContent = UrlFetchApp.fetch(csvUrl).getContentText();
  var csvData = Utilities.parseCsv(csvContent, ";");

  // Retrieve Spreadsheet and put the CSV data.
  var root = DriveApp.getFoldersByName("Produits");
  while (root.hasNext()) {
    var folder = root.next();
    var files = folder.getFiles();
    while (files.hasNext()) {
      var spreadsheet = SpreadsheetApp.open(files.next());
      var name = spreadsheet.getName().toUpperCase();
      var values = csvData.reduce((ar, r) => {
        if (!keywords.some(e => r[2].includes(e)) && r.join("").toUpperCase().includes(name)) {
          ar.push(r);
        }
        return ar;
      }, []);
      if (values.length == 0) continue;
      var sheet = spreadsheet.getSheets()[0];
      sheet.clearContents().getRange(1, 1, values.length, values[0].length).setValues(values);
    }
  }
}
  • In this modification, the CSV data is retrieved at the outside of the while loop. Using the retrieved CSV data and each Spreadsheet name, the data is put to the 1st tab of each Spreadsheet.

References:

Tanaike
  • 181,128
  • 11
  • 97
  • 165
  • Thank you very much @Tanaike this is already better than what I did apparently. I tried with the sample file and it works fine but weirdly when I use a larger file the data is not being parsed (the script runs without error but there is not data in the spreadsheet afterwards) I have created a new sample file to see the problem if you may : https://incensy.tempurl.host/test-ct-2.csv - This file has more columns but I do not see why it could be a problem – Antoine May 05 '22 at 07:08
  • @Antoine Thank you for replying. I apologize for the inconvenience. About your new issue, I think that the reason for your issue is due to the structure of the CSV data is different from your 1st question. So, in order to check all columns of the CSV data, I updated my proposed script. Could you please confirm it? If that was not useful, I apologize again. – Tanaike May 05 '22 at 07:23
  • I would like to give you warm thank you for this @Tanaike , I have been trying for a long time and you found a great solution. If I may add a related question, if you have the time : I would like to be able to skip (do not parse) lines if they contain a specific word. I have added my code bellow but it does not work properly, do you know a better way to do that please ? – Antoine May 05 '22 at 09:01
  • Here is the code related to the comment above : `var rows = sheet.getDataRange(); var numRows = rows.getNumRows(); var values = rows.getValues(); var rowsDeleted = 0; for (var i = 0; i <= numRows - 1; i++) { var row = values[i]; if (row[3] == 'Coque' || row[3] == 'chargeur') { sheet.deleteRow((parseInt(i)+1) - rowsDeleted); rowsDeleted++; } }` – Antoine May 05 '22 at 09:02
  • @Antoine Thank you for replying. I'm glad your issue was resolved. About your new question, I would like to support you. But, I have to apologize for my poor English skill. Unfortunately, I cannot understand your new question of `I would like to be able to skip (do not parse) lines if they contain a specific word. I have added my code bellow but it does not work properly, do you know a better way to do that please ?`. Can I ask you about the detail of your new question? – Tanaike May 05 '22 at 09:05
  • I will try to explain better, in the CSV data there are data (rows) I do not want to import to my spreadsheets. I would like to do something like if column 3 contains 'this' then do not import this row. – Antoine May 05 '22 at 09:09
  • @Antoine In your initial question, you said `I would like the data to be cleared and replaced by the new data everyday.`. If you want to keep this, about your new question, you want to put the data by filtering with both the Spreadsheet name and other keywords. And, from your showing script, you want to filter with the column "C" for other keywords. Is my understanding correct? – Tanaike May 05 '22 at 09:10
  • Yes this is correct @Tanaike , I apologize for not explaining clearly enough. – Antoine May 05 '22 at 09:13
  • @Antoine Thank you for replying. I thought that I could correctly understand it. So, I updated my answer. Could you please confirm it? In this modification, the rows including the values of `keywords` are not put. – Tanaike May 05 '22 at 09:15
  • @Antoine As an additional information, in this modification, for example, when `var keywords = ["sample1", "sample2"];` is used, when `sample1` OR `sample2` in the column "C" are included in the row, the row is not put. If you want to put the rows not including both `sample1` AND `sample2`, please modify `!keywords.some(e => r[2].includes(e))` to `!keywords.every(e => r[2].includes(e))`. – Tanaike May 05 '22 at 09:19
  • Once again @Tanaike your code is working like perfection, just a little thing, is it possible to make it not case sensitive. For example to exclude "sample1" but also "Sample1" and "SAMPLE1" ? – Antoine May 05 '22 at 09:26
  • @Antoine Thank you for replying. About your additional question of `For example to exclude "sample1" but also "Sample1" and "SAMPLE1" ?`, please modify `!keywords.some(e => r[2].includes(e))` to `!keywords.some(e => r[2].toUpperCase().includes(e.toUpperCase()))` and test it again. – Tanaike May 05 '22 at 09:29
  • This is amazing @Tanaike, I can't express how grateful I am. Thank you very much, you made my day ! – Antoine May 05 '22 at 09:34
  • @Antoine Thank you for replying. I'm glad your issue was resolved. Thank you, too. – Tanaike May 05 '22 at 11:32
  • Hi @Tanaike, I have been adding some stuff to the code your provided (still learning) but could not make it as fast as it was. If you have some time and are willing to check the new post I will surely appreciate it very much !https://stackoverflow.com/questions/72582826/make-appen-fonction-work-faster-in-google-google-sheet/72583139 – Antoine Jun 11 '22 at 09:37