2

We have been working on a web service (http://www.genomespace.org/) that allows computational biology tools that implement our REST API to, among other things, read and write files stored on "The Cloud".

Our default file storage is a common Amazon S3 bucket, but we now allow users to mount their own private S3 bucket as well as files on Dropbox.

We are now trying to enable similar functionality for Google Drive and have run into some problems unique to Google Drive that we have not encountered with S3 or Dropbox.

  1. Only way to allow clients that are not Google-authenticated to read files unobtrusively is to make the files "Public". Our preference would be that once the user has authorized access to our application via OAuth2, the user files could remain "Private" in Google Drive.
    However, even though the user has already authorized our web service to offline access to their "Private" files, we have not found a way to generate a URL that a client authorized by our system can use to GET the file directly without the client being logged into Google as well. The closest we have come to this functionality has been to change the file permissions to "Anyone with Link", except that for files greater than 20MB Google insists on returning an intermediate web page warning that the file has not been scanned for viruses. In addition to having to mess with file permissions, this would break our existing clients. Only when the file is "Public" and we utilize URLs of the form https://googledrive.com/host/PARENT_FOLDER_ID/FILENAME can non-Google clients read the files without interference.

  2. Have not found any way for clients that are not Google-authenticated to upload a file to Google Drive. Our API allows our authorized clients to PUT files directly to the backing file storage using URLs provided by our server. However, even if a folder is marked "Public", the client requires Google authentication credentials to save to Google Drive. We could deal with both of these issues with intermediate hops through our system (e.g., our web server would first download the file from Google Drive and then allow the client to GET it) however this would be woefully inefficient and, hopefully, unnecessary. These problems have been discussed multiple times before on stackoverflow (e.g. here and here and have read the responses very carefully, but have not seen any recent discussion.

The Google folks direct their API users to post on stackoverflow for support, so I am hoping for a fresh look from insiders.

Community
  • 1
  • 1
MarcoO
  • 21
  • 3
  • I am asking a question based on the problems I have encountered using the Google Drive API. It could very well be that my problems can only be solved by extending the existing API. Or it could be that there is some solution or workaround that I am not aware of. I just don't know. – MarcoO Jun 16 '14 at 18:17
  • This all raises the question of why do you have unauthenticated clients in the first place? In #1 it sounds like your preference is to have files remain private. That naturally requires authentication. Likewise, uploading requires *somebody* own the data, which again requires authentication. So what architectural constraint is at play here that you can't send along access tokens? – Steve Bazyl Jun 16 '14 at 21:41
  • You could use a service account. Files would then be uploaded to the service account. you would then be able to programmatically use the service account to then allow your users to download them again. https://developers.google.com/accounts/docs/OAuth2ServiceAccount – Linda Lawton - DaImTo Jun 17 '14 at 08:05
  • The clients are unauthenticated from the Google Drive perspective, not from our system's perspective. We know who they are. We just don't want to have the user create an account and login into our system and then have an account with Google and login again. – MarcoO Jun 17 '14 at 17:20
  • If I understand correctly the Google Drive security system, our app is able to reach "Private" files when granted offline access by the user. Like a couple of commenters have suggested, our web server could indeed download the files to an intermediate location and then let the client get it from there. It would be more efficient to have the clients communicate with the Google servers directly. Google has a better server infrastructure than we do. – MarcoO Jun 17 '14 at 17:48
  • What we are looking to do is to have our application do authorization for access to the Google Drive files and then to empower to directly communicate with the Google servers for file downloads and uploads, without our servers having to deal with the bytes contained in those files. In other systems there are schemes where the application can generate temporary signed URLs to GET or PUT a file. Another scheme I have seen is that the application can create a transient user account with very specific and narrow permissions for the file operation that it is authorized to do. – MarcoO Jun 17 '14 at 17:49
  • Steve Bazyl mentioned passing along access tokens to the client. Perhaps that is what we should be doing. Is there a way to generate access tokens that can safely be passed to a client that are either temporary or narrowly constrained? – MarcoO Jun 17 '14 at 17:50

1 Answers1

1

The general answer is: dont make the drive requests through the user's browser. Insead do everything from your servers. You are the one having the (refresh) tokens for users, so you should make all requests like a proxy between the user and Drive. Same for downloading, you download it and return to the user. As long as you use each drive's token there shouldnt be rate limit/quota issues.

Zig Mandel
  • 19,571
  • 5
  • 26
  • 36