0

I am developing a project in Google Cloud using both their App Engine and Compute Engine. I have a virtual machine instance set up on Compute Engine, with the name "instance-1". On this instance is the python file (file.py):

name = '<REPLACE_WITH_YOUR_NAME>'

print(name)

Well, this isn't exactly the file.py, but the concept applies below. Additionally, I have an App Engine project written in NodeJS, which is connecting to this instance via Google's Compute Engine API. Here is what I have in regards to that:

const Compute = require('@google-cloud/compute');

const compute = new Compute();
const zone = compute.zone('us-east1-b');
const vm_name = 'instance-1';
const vm = zone.vm(vm_name);
const my_name = "David Weiss"

// TODO: insert the variable my_name into the python code file.py where it says '<REPLACE_WITH_YOUR_NAME>'
// In my head, it would look something like this: vm.getFile('file.py', 'write').replace('<REPLACE_WITH_YOUR_NAME>', my_name);

After getting the instance 'instance-1, I don't know how to modify (or even add/replace/delete) files on it using NodeJS and the Compute Engine API. Can this be done? If it's not possible to replace the text within file.py, I would be okay with deleting the entire file and just writing a brand new file with my_name already inserted in there.

David Weiss
  • 93
  • 1
  • 5
  • You cannot directly modify files on Compute Engine using the API. You can use SSH/SFTP for file transfer. A simpler strategy is to deploy to GitHub or Cloud Storage and pull the files to Compute Engine in your startup file which runs on startup/reboot. Also, Cloud Build is designed for building and deploying apps. – John Hanley Mar 17 '20 at 00:49

1 Answers1

1

Think of your Compute Engine instance as a "computer". If you had files on a PC, how would you modify the files on that PC remotely? The GCP API for Compute Engines doesn't give you access to the file system of that instance. Instead, you would have to use technology such as scp or ftp. Perhaps if you described the higher level story, there might be alternative concepts we could use. For example, when a Compute Engine boots, it can run a startup script that might copy from someplace (eg. GCS). Another possibility is that your compute engine could run an app itself against which you could make REST requests and pass data that would be the content of a file written by the compute engine app.

If you still want to go down the file copy route and you want the requesting app to be written in python, then a possibility might be to review:

How to scp in Python?

2020-03-17 - Based on more comments

For provisioning new instances of Compute Engine ... I am sensing that you are using GCP API to create new Compute Engine instances through your App Engine app. If the puzzle you pose were before me, I'd be thinking down the following lines.

When the App Engine app determines that a new Compute Engine instance is to be created, we obviously have to give that new instance a unique name. No two Compute Instances can have the same identity. We thus have a "key" for that instance. Next, I would have the App Engine app create a file in Google Cloud Storage (GCS) that contains the exact file that you want inside the Compute Engine. Your app could build the content of the file dynamically. There would be one file per compute engine instance and the file name would match the name of the compute engine instance. At this point, we have your desired file in GCS. Next, I would create a shell script that copies the file from GCS (based on the name of the compute engine in which the script runs) to the local compute engine instance file system. Finally, I would specify this script as a "startup script" that is executed when the Compute Engine boots.

When the compute engine DOES boot, it will run the startup script early on in the boot cycle but prior to the user being able to login. The script would copy the file from GCS to local file system storage and when the script completes, the user can login and will find the file.

See also:

Kolban
  • 13,794
  • 3
  • 38
  • 60
  • Very interesting, thank you! Let me describe more of the project and my thinking. I started off with App Engine to create a NodeJS application that provides a front-end to my users. I wanted my users to input some data, let's say their email address, and then click a button to programmatically create their own personalized instance on Compute Engine. On their instance would be a python file.py that would be scheduled through cron to run from 8am to 6pm every day and it would make use of the email supplied via the front end application. All good, but what if the usr needed to change that email? – David Weiss Mar 17 '20 at 15:05
  • Howdy David ... I'll update the answer with more thoughts ... some questions for you first. Is your goal to allow end users to have their OWN Compute Engine instance? What I think I'm hearing you say is that you want to provision a Compute Engine instance on demand for a user through an App Engine but have that instance customized for the user? Does each user have their own Compute Engine or are you trying to share one Compute Engine across multiple users? – Kolban Mar 17 '20 at 15:53
  • Yes, your understanding is correct. I would like users to provision their own Compute Engine instance on-demand and personalized with their own data running on the files stored on that instance. So, each user would be matched to their own instance (on a 1:1 ratio). This makes sense in my head but if there's a better way, I am open to hearing it! Also, the file structure in each user's instance would be the same, the only difference being the data arguments used to run the functions defined in each instance's file.py. For instance, user 1 would use email_1 to run their file.py, user 2, email_2. – David Weiss Mar 17 '20 at 16:22