File downloading function takes more than 6 minutes, how can I improve it, in google app script?

Question

My file downloading function takes more than 6 minutes. Therefore, I couldn't download the whole files. Is there anyway to complete the download within 6 minutes? Or any other solution ? Html

<html>
    <form id="downloadpdf">
        <input id="urlclass" type="text" name="urlname"  />
    </form>
    <script src="//ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script>
    <script>
        var urllink = [];     //this value assigned by another function. But let us assign statics variable as example urllink={“url1,url2,url3”}
        $(document).ready(function() {
            $("#downloadpdf").submit(function() { 
                $("#urlclass").val(urllink);

                google.script.run.withSuccessHandler(function(retsearch){ 
                }).downloadthefile(this);
            });
        });
    </script>
</html>

code.gs

function downloadthefile(urlpass) {
    var timezone = SpreadsheetApp.getActive().getSpreadsheetTimeZone();
    var now = Utilities.formatDate(new Date(),timezone, "EEE MMM d yyyy HH:mm:ss");
    var dest_folder = DriveApp.createFolder(now);
    var dest_folder_id = DriveApp.getFoldersByName(dest_folder).next().getId();
    var source_folder = DriveApp.getFolderById(“the source folder id");
    var dest_folder = DriveApp.getFolderById(dest_folder_id );

    // The url that is passed to the function as string  
    var urlstring = urlpass.urlname;                 
    // Make array of url 
    var resultarray = urlstring.split(',');                

    for(var i=0; i < resultarray.length; i++){
        var fileurl = resultarray[i];

        // Retrieve the url Id
        var fileid = fileurl.match(/[-\w]{25,}/);
        var file = DriveApp.getFileById(fileid);
        var filecopy = file.makeCopy(file.getName());
        dest_folder.addFile(filecopy);
    }
}

The connection is good.But the file is a bit big. @Jaromanda — Jona, Jul 06 '17 at 02:36
The url length could be 200,300 or more.The length is not fixed. It depends by another function. @BKSpurgeon — Jona, Jul 06 '17 at 02:37
ahh, it's "good", is that in megabits per second? and "bit big" ... is that in megabytes? — Jaromanda X, Jul 06 '17 at 02:37
@user7428112 when i wrote the simplest of all hello world using google script it did take about 1-2 seconds. so imagine if you have 200-300 large files that are being copied over in the same thread......that's definitely gonna take some type. perhaps you can upgrade your google account for better performance? — BenKoshy, Jul 06 '17 at 02:39
I wish if I could upgrade my google account. But that is the management body decision. Isn't it possible to continue the script from where I stop after 6 minute or optimize the code to run under 6 minute ? @BKSpurgeon — Jona, Jul 06 '17 at 02:46
(1) you'll need to keep a log of which files are created and then resume functions after 6 minutes, or (2) perhaps investigate some ideas here: http://ramblings.mcpher.com/Home/excelquirks/parallel - apart from that i'm out of ideas. I don't think it's possible to optimise the algorithm any further, but perhaps instead of copying each individual separate file, you can simply copy a few folder etc instead? — BenKoshy, Jul 06 '17 at 03:26
I tired to modify the above code by adding trigger based on this example https://stackoverflow.com/questions/41971806/how-can-i-increase-the-6-minute-execution-limit-in-google-apps-script. But it doesn't work. How can I modify the code by adding a trigger? @ BKSpurgeon — Jona, Jul 06 '17 at 09:02
Please look at the answer on this question https://stackoverflow.com/questions/44847759/how-do-i-extend-script-execution-time-google-script/44853845 You may want to use XHR requests to Google Drive API in your modal window html file to have no limits on execution time. — Kos, Jul 06 '17 at 09:13
I second Kos answer. You have to circumvent apps script all together and call the Google Drive API directly from the client html file via XHR. That way you're not bound by the 6-minute execution limit imposed on Apps Script functions. — TheAddonDepot, Jul 06 '17 at 11:41
I tried to understand your example and covert it to my case. But I couldn't figure out how I could apply XHR request in my case. Is it possible to make a show case on my code please ? Thank you ! @Kos — Jona, Jul 10 '17 at 02:02
I was trying to use trigger to workaround this problem which is the execution time is more than 6 minutes. But my function has an argument and I can't pass arguments to the function if I call them by trigger. What can I do? — Jona, Jul 10 '17 at 06:41

Kos · Accepted Answer · 2017-07-18T11:24:07.957

Workaround for not hitting Google Apps Script execution time limit is to run copy requests in your modal window code (plain JavaScript with XHR requests to Google Drive API) instead of doing it in GAS script.

Edited [adapted for Deploy as web app]:

Code.gs:

function doGet(){
  var template = HtmlService.createTemplateFromFile('copyDriveFiles')

  // need to have next line of text somewhere (even commented out) to trigger correct scopes for script and token:
  // DriveApp.getFiles() // <- DO NOT DELETE THIS COMMENT

  // pass token
  template.data = {
    token: ScriptApp.getOAuthToken()
  };

  var html = template.evaluate().setTitle('Download');

  return html;
}

copyDriveFiles.html:

<html>
<head>
</head>
<body>

<form id="downloadform">
   <input id="downloadpdf" type="submit" value="download">
</form>

<script src="//ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script>
 <script>
        // this value assigned by another function. But let us assign statics variable as example urllink = '{“url1,url2,url3”}'
        var urllink = 'https://docs.google.com/document/d/XXXXXX/edit,https://docs.google.com/document/d/ZZZZZZ/edit';

        // download files
        $(document).ready(function(){
          $("#downloadform").submit(function(e){
            downloadFiles(urllink);  

            return false;
          });
        });

        function downloadFiles(urllink)
        {
          <?
            // create destination folder
            var timezone = Session.getScriptTimeZone();
            var now = Utilities.formatDate(new Date(), timezone, "EEE MMM d yyyy HH:mm:ss");
            var folder = DriveApp.createFolder(now);
            var folderId = folder.getId();
          ?>

          // get folder id
          var folderId = <?=folderId?>;

          var resultArray = urllink.split(','); 

          // loop file urls
          for (var i = 0; i < resultArray.length; i++)
          {
            // retrieve the file id from url
            var fileId = resultArray[i].match(/[-\w]{25,}/);

            // copy file to folder
            copyFileToFolder(fileId, folderId);
          }
        }

        function copyFileToFolder(fileId, folderId)
        {
            var xhr = new XMLHttpRequest();

            // request payload - set parents
            // here you can specify file name etc: https://developers.google.com/drive/v3/reference/files/copy
            var requestBody = {
              parents: [folderId]
            };

            xhr.open('POST', 'https://www.googleapis.com/drive/v3/files/'+fileId+'/copy');
            xhr.setRequestHeader('Authorization', 'Bearer <?=data.token?>');
            xhr.setRequestHeader('Content-type', 'application/json');

            // send request
            xhr.send(JSON.stringify(requestBody));
        }
    </script>
    </body>
</html>

Result:

Thank U! I want to use for google app script, I changed code.gs function by function doGet(){ var html = HtmlService.createTemplateFromFile('copyDriveFiles').evaluate() .setTitle('Download').setSandboxMode(HtmlService.SandboxMode.NATIVE); return html; } Added following code on html

$(document).ready(function() { $("#downloadpdf").submit(function() { downloadFiles(urllink); }); }); I got error @kos — Jona, Jul 11 '17 at 01:53
The error is: "The functionality requested a security token, but the project does not use any services that require a security token. (line 2, file "Code", project "download no limit") " @Kos — Jona, Jul 11 '17 at 10:44
Can you show full code, as I cannot get this error for code you wrote in comment. — Kos, Jul 12 '17 at 15:23
I shared the code here https://docs.google.com/spreadsheets/d/1Ymyzcd5dLy5iDuTnykQnhHpK4sZDH7Sezu3l4vmsfUQ/edit#gid=0 @Kos — Jona, Jul 18 '17 at 00:58
I got the following error, you dont have permission to call getActive, I shared the link here https://script.google.com/macros/s/AKfycbyKXqf6KEaF75EH02ZPJnDrIH7QshJacJQqwR5ZJY2r3nu1r3Y/exec @Kos — Jona, Jul 18 '17 at 11:24

File downloading function takes more than 6 minutes, how can I improve it, in google app script?

1 Answers1