5

NOTE: this question was flagged as a posible duplicate of 13413036. This is not the case, in that question, they are asking how to upload a file using resumable upload. This code does exactly that, and it works. Only problem is that the library crashes after prolonged period of time.

I am trying to upload a 40GB file from a Cloud Compute instance to Drive.

I have a PHP script running at the command line which crashes after about an hour or so.

PHP Fatal error:  Uncaught exception 'Google_Service_Exception' with message 'Error calling PUT https://www.goog
leapis.com/upload/drive/v2/files?uploadType=resumable&upload_id=AEnB2UrkY_dh7-GmPp33joiPzhBT3DHY6fg4bgN_PNWVgEYg
IBZmFf0UceJbLS1XeyaJ0bFxdMTNWtPVEAQKIf8uE7HBExuG6g: (503) Service Unavailable' in /var/www/backend/phpsdk/exampl
es/apiClient/src/Google/Http/REST.php:110
Stack trace:
#0 /var/www/backend/phpsdk/examples/apiClient/src/Google/Http/MediaFileUpload.php(183): Google_Http_REST::decode
HttpResponse(Object(Google_Http_Request), Object(Google_Client))
#1 /var/www/backend/phpsdk/examples/drive.php(95): Google_Http_MediaFileUpload->nextChunk('\xBA\x9AX\xBA\xAB\x9C
C\x91\x9E1\xDB|\xD0\xD5\xFB...')
#2 {main}
  thrown in /var/www/backend/phpsdk/examples/apiClient/src/Google/Http/REST.php on line 110

If I execute the script again, then it creates a new file in Drive.

How can I resume the initial upload after a crash? Can I set resumeUri after initializing Google_Http_MediaFileUpload ? How do I get PHP to skip reading the file to a specific point?

The code is copy/paste from the example:

  // Create a media file upload to represent our upload process.
  $media = new Google_Http_MediaFileUpload(
      $client,
      $request,
      mime_content_type(TESTFILE),
      null,
      true,
      $chunkSizeBytes
  );
  $media->setFileSize(filesize(TESTFILE));
  // Upload the various chunks. $status will be false until the process is
  // complete.
  $status = false;
  $handle = fopen(TESTFILE, "rb");
  while (!$status && !feof($handle)) {
    $chunk = fread($handle, $chunkSizeBytes);
    $status = $media->nextChunk($chunk);
  }
  // The final value of $status will be the data from the API for the object
  // that has been uploaded.
  $result = false;
  if ($status != false) {
    $result = $status;
  }
  fclose($handle);
}
miturbe
  • 715
  • 4
  • 17
  • possible duplicate of [Google Drive API - PHP Client Library - setting uploadType to resumable upload](http://stackoverflow.com/questions/13413036/google-drive-api-php-client-library-setting-uploadtype-to-resumable-upload) – Linda Lawton - DaImTo Jul 03 '15 at 07:22
  • Not a duplicate. In that question, they are asking how to upload a file using resumable upload. This code does exactly that, and it works. Only problem is that the library crashes after prolonged period of time. – miturbe Jul 03 '15 at 16:24

2 Answers2

0

It's not a duplicate, it's a legit bug. I have the same problem and tried posting in the github and the main developer said to do a try catch for the nextChunk thing to see if you can resume it after an exception has been caught. I stopped the development for that module so far, but I think the try/catch may solve it unless the API is made to prevent big sized files from being uploaded. Mine is 17ish GB, and I made it to 12 GB before it stopped, not always stopping in the hour mark, although it also stopped in the hour mark. Wanted to comment but no rep enough yet.

Rg14
  • 300
  • 1
  • 14
  • In the end, I fixed it by sending bigger chunks, since I had RAM available I set the chunkSize to 20 * 1024 * 1024 and it uploaded in 40 minutes or so. – miturbe Jul 03 '15 at 16:22
  • Oh, I thought it could only handle 1*1024*1024 sizes, as it's what it says in the Drive example. Let me try it out. – Rg14 Jul 03 '15 at 16:51
  • If you don't mind me asking, how many RAM and upload speed do you have? I just want to compare to see if I can run it like you too... – Rg14 Jul 03 '15 at 17:13
  • 1
    the server has 12GB of RAM. and the migration is within the Google network. Cloud Compute instance and Google Drive. However, speeds were only at around 22MBps – miturbe Jul 04 '15 at 03:03
0

In the end, I fixed it by sending bigger chunks, since I had RAM available I set the chunkSize to 20 * 1024 * 1024 and it uploaded in 40 minutes or so.

the server has 12GB of RAM. and the migration is within the Google network. Cloud Compute instance and Google Drive. However, speeds were only at around 22MBps

miturbe
  • 715
  • 4
  • 17