0

I would like to get data with GAE (free app) and then store it. I don't need to use this data within GAE - once collected, there should be exported and used locally (with 3rd party software). Is there any alternatives to what Google suggests? Currently I am over GAE quota (write database ops).

Probably, I could write the data directly to some txt/csv file and then download it? Or, write it to Google Docs spreadsheet?

Upd. Each day I will need to add new records at the end of txt/csv file. So, it will not be one time file upload.

LA_
  • 19,823
  • 58
  • 172
  • 308

2 Answers2

1

Using the drive API you can generate a spreadsheet. like you suggest.

Upload CSV to Google Drive Spreadsheet using Drive v2 API

Or use the spreadsheets API

https://developers.google.com/google-apps/spreadsheets/

Generally Google Cloud Storage should be used for this kind of thing but if you are trying to avoid paying anything then you may have to settle

Community
  • 1
  • 1
Bachmann
  • 748
  • 6
  • 12
  • The example you gave is for one file, but I will add records to the file each day.. – LA_ May 20 '14 at 20:22
  • You can also add to an existing spreadhseet with the Spreadsheets API https://developers.google.com/google-apps/spreadsheets/ – Bachmann May 20 '14 at 20:30
  • I am afraid it will not work with Google service account (provided by GAE), which means that I will have to authorize the app thru OAuth manually... – LA_ May 21 '14 at 14:21
  • Thats no good. You can try grabbing the auth header from an oauth2 request and adding it to the oauth1 request. Like this question suggests http://stackoverflow.com/questions/21467305/how-to-authenticate-with-google-spreadsheets-api-using-python-client-discovery – Bachmann May 21 '14 at 14:40
  • Finally, I've decided to save the data to Fusion Tables. First of all, it is free, secondly it is supported by Google API Python Client, which I used in another problem. And, I can access Fusion Tables with Service account (looks like service account will work with Spreadsheet also, but I haven't tried). – LA_ May 22 '14 at 12:22
1

If you don't need data replication and other features of the Datastore, there is no reason to use it.

You can store txt or CSV files in Google Cloud Storage, for example. You get 5GB of free storage, which is a lot of data.

Andrei Volgin
  • 40,755
  • 6
  • 49
  • 58
  • Is there any easy way to add records to the existing file stored at Google Cloud Storage? – LA_ May 20 '14 at 20:22
  • You have to read the file, update it, and save it back. A more efficient approach is saving data in batches - for example, one day of records per file, instead of appending new records to the existing file. Use the date or timestamp to name these files - it will make processing them much easier. – Andrei Volgin May 20 '14 at 20:25
  • Couldn't I write data just to some GAE file (without using other Google services)? – LA_ May 20 '14 at 20:26
  • There if no file system in GAE. Writing to the Google Cloud Storage is very simple. – Andrei Volgin May 20 '14 at 20:33