3

I'm developing a web application that's going to start with 200gb of data to be storaged. Over the years, the same application possibly can reach 1tb, perhaps 2tb in 5 years.

What I want from this application is the clients to upload files to the server and the server then upload files do Google Drive, persisting the webviewlink on database. It's working this way on localhost.

I know two options for authentication for Google Drive API: client account and service account. Service Account's option fits better for me because I want the server to have control of the files, not the client have control. But Service Account can storage too few data and the storage limit can't be increased. The limit is something around 15gb I guess, not sure.

If the Service Account will not help me, what options would I have to storage 2tb of data or more? Should I find another way to storage the files?

I'd like to stay using Google. If there's not any option using Google Drive API, please, suggest anything else for this scenario.

James Z
  • 12,209
  • 10
  • 24
  • 44
Robson Faxas
  • 33
  • 2
  • 5
  • You may have to pay google fees for their Google Cloud so that you can get access to a larger volume of storage. I believe their free accounts are limited to 15GB, whereas a business/paid account has a much higher storage limit. – Azeame Jan 26 '19 at 22:45
  • @Azeame Paying Google Cloud for my account, the Service Account (used for Google Drive API) limit would be increased or I'd have to use another totally different API to work with uploads and downloads on Google Cloud? – Robson Faxas Jan 26 '19 at 23:05
  • I have a service account which has uploaded nearly 200GB which has been backing up data for a couple years, but I can find anything regarding quotas or why I am able to pass your 15GB to help you. – Azeame Jan 26 '19 at 23:13
  • It'd be perfect if you confirm me what's the reason that your service account is able to have more than 15gb, I'd be grateful. It'd be also an answer for this question. – Robson Faxas Jan 26 '19 at 23:33
  • I looked into it some more and it appears that I was mistaken, I actually have separate installs of the same webs service, I thought they had separate users service accounts only for the calendar but that's not the case those separate service accounts also manage the drive backups. As far as I can see, there may actually be a 15GB/service account limit to contend with. – Azeame Jan 26 '19 at 23:50
  • What a pity. Anyway, I submitted a question to Google Cloud about the Google Cloud Storage. I'm thinking about this service. While I'm not sure what to do yet, I'm going to keep this question opened to the next people willing to help like you. Thanks @Azeame – Robson Faxas Jan 27 '19 at 00:04

1 Answers1

2

You have a couple of options.

  1. Use a regular account instead of a Service Account. You will still need to pay for the storage, but it will work and you'll have everything in a single account. From your question "I want the server to have control of the files, not the client have control" I suspect you have looked at the OAuth quickstart examples and concluded that only end users can grant access. That's not the case. It's perfectly valid, and really quite simple, for your server app to grant access to an account it controls. See How do I authorise an app (web or installed) without user intervention? for how to do this.

  2. Use multiple Service Accounts and shard your data across them. The various accounts could all share their folders to a kinda master account which would then have a coherent view of the entire corpus.

Personally I'd go with option 1 because it's the easiest to set up and manage.

Either way, make sure you understand how Google will want to charge you for the storage. For example, although each Service Account has a free quota, it is ultimately owned by the regular user that created it and the standard user quota limits and charges probably apply to that user.

pinoyyid
  • 21,499
  • 14
  • 64
  • 115
  • 2
    I see that I have some materials to read about now. I confess that I read your answer but, due to my ignorance, I thought it wasn't an official process, like something uncertain for the future. Now I read it better, watched the video and I realized that it's official and it looks really possible for my application. Thanks for the suggestion, I'll try the first option next week and write here a feedback. – Robson Faxas Jan 27 '19 at 15:07
  • You're not alone. All of the Google documentation describes OAuth being used for end user auth, probably because it is the more difficult use case. Then it gets made worse by leaving the impression that Service Accounts are the only mechanism for server access. Feel free to ping me with any questions as you go fwd. – pinoyyid Jan 27 '19 at 17:05
  • 2
    It worked! Thank you very much Pinoyyid, you saved my project! – Robson Faxas Feb 13 '19 at 22:55