0

I am working on a project that involves using an Google App Engine (GAE) server to control one or more Google Compute Engine (GCE) instances. The rest of the project is working well, but I am having a problem with one specific aspect: file management. I want my GAE to edit a file on my GCE instance, and after days of research I have come up blank on how to do that.

The most straightforward example of this is:

Step 1) User enters text into a GAE form.

Step 2) User clicks a button to indicate they would like to "submit" the text to GCE

Step 3) GAE replaces the contents of a particular (hard-coded path) text file on the GCE with the user's new content.

Step 4) (bonus step) GCE notices that the file has changed (either by detecting a change or by way of GAE alerting it when the new content is pushed) and runs a script to process the new file.

I understand that this is easy to do using SCP or other terminal commands. I have already done that, and that works fine. What I need is a way for GAE to send that content directly, without my intervention.

I have full access to all instances of GAE and GCE involved in this project, and can set up whatever code is needed on either of the platforms.

Thank you in advance for your help!

aroarus
  • 3
  • 2

4 Answers4

1

You could use the Google Cloud Storage Client to load the files into a Cloud Storage bucket, and have the GCE instances copy files out of GCS and onto the local filesystem. On the GCE side, you could use Object Change Notification to achieve your bonus step 4.

CharlesB
  • 227
  • 1
  • 1
  • This seems to do exactly what I need! Thank you! – aroarus Jun 13 '14 at 23:37
  • Charles, even better you can mount the Cloud Storage bucket locally into /mnt/mybucket, so you wouldn't need any notification whatsoever (unless GCE vm needs to update other stuff depending on this, in which case your solution is probably the best). – Riccardo Aug 07 '14 at 13:28
0

The most straightforward approach seems to be:

  1. A user submit a form on App Engine instance.
  2. App Engine instance makes a POST call to a handler on GCE instance with the new data.
  3. GCE instance updates its own file and processes it.
Andrei Volgin
  • 40,755
  • 6
  • 49
  • 58
  • Thank you! This is what I am trying to do. I'm worried about using the POST call when the amount of data to be transferred gets very large. Is that an issue with POST? – aroarus Jun 14 '14 at 00:03
  • You can configure the limit on your own GCE instance. It can easily be in tens of MBs. I also suggested another solution - see my other answer. – Andrei Volgin Jun 14 '14 at 00:11
0

You can set-up a cron in GAE

https://developers.google.com/appengine/docs/python/config/cron?hl=es

and create a SCP job directly in your code.

How to scp in python?

i hope this work for you.

Community
  • 1
  • 1
Carlos Rojas
  • 5,547
  • 2
  • 14
  • 13
0

You can set an action URL in your form to point to the GCE instance (it can be load-balanced if you have more than one). Then all data will be uploaded directly to the GCE instance, and you don't have to worry about transferring data from your App Engine instance to GCE instance.

Andrei Volgin
  • 40,755
  • 6
  • 49
  • 58