0

Within a VM, I have a container with a simply Python script I'd like to run via a Cloud Function. I understand how to start/stop the VM, but I cannot figure out how to run the script via the Cloud Function. Is it possible to do this via a Cloud Function?

Here are the details:

This is the GCP Cloud Function which starts a VM, with code using the Python SDK.

from googleapiclient import discovery

service = discovery.build('compute', 'v1')

# Start the VM

# Project ID defined here
project = '' 
zone = 'us-central1-a'  
# name of the VM
instance = 'my-instance'

request = service.instances().start(project=project, zone=zone, instance=instance)
response = request.execute()

print("success!")

This VM my-instance was deployed with a Docker container (see here for details).

Within the VM, I can use the docker container as I normally would, e.g. with docker run -it container-name python3 myscript.py. (For the purpose of this question, it's exactly like running a script which prints "hello world" in python, as explained in this link)

How can I simply execute something like docker run -it my-container python3 my-script.py?

Is it possible to set up docker run to be executed when the instance is launched?

EB2127
  • 1,788
  • 3
  • 22
  • 43
  • 1) Your goal might require more resources/capabilities than Cloud Functions provides. Consider Cloud Run. Note that I am not sure and I have a positive bias towards using Cloud Run for these types of tasks. 2) Once you start the VM, you can use an SSH library to connect to the VM and run your Docker command. This is very easy to do in Python and most other languages. – John Hanley Mar 02 '21 at 05:14
  • Hi @JohnHanley, thanks for the help. I've considered Cloud Run, but my understanding is that there's a limit to RAM that the container uses (8GB I think). If more RAM was accessible, then I agree that Cloud Run would fit this task. – EB2127 Mar 02 '21 at 05:28
  • I am not sure I follow you. Cloud Run is the big brother to Cloud Functions. And you do not need 8 GB of RAM to create an SSH connect to execute a remote command. – John Hanley Mar 02 '21 at 05:34
  • I could be confused; your input would be helpful. For Cloud Run, within the container (e.g. running the python code), is there a memory limit? https://cloud.google.com/run/docs/configuring/memory-limits My understanding is that it is 8GB. If this is correct, then I need to run the docker containers within a VM with more resources than this. Does this make sense? – EB2127 Mar 02 '21 at 05:38
  • Your goal is to start a VM and then run a Docker container in the VM. I am talking about the part that starts the VM and remotely starts the Docker container. My recommendation is to replace Cloud Functions with Cloud Run. – John Hanley Mar 02 '21 at 05:50
  • You're saying using Cloud run to start the VM? Again, I could be confused---the processes via the container will need more than 8GB to run. – EB2127 Mar 02 '21 at 05:54
  • 1
    The answer that guillaume posted is very good. However, if your new goal is to run Docker from a Compute Engine startup script, I recommend instead to use Container Optimized OS which is designed to run a container. https://cloud.google.com/container-optimized-os/docs – John Hanley Mar 02 '21 at 22:23

1 Answers1

2

To execute something when the VM starts, you can use startup scripts. however, don't use the command with -it it's for interactive mode; simply perform a docker run in the startup script.

Just after, you can write a code to stop automatically the VM when the docker run command is over.

guillaume blaquiere
  • 66,369
  • 2
  • 47
  • 76
  • Thanks for this! I didn't realize startup scripts existed! – EB2127 Mar 02 '21 at 14:29
  • Given you're answering the other question I had, couldn't the Cloud Function detecting the file upload simply pass this filename to the VM which is started? The file `filename.csv` uploaded to the Storage bucket could be read by `python3 my-script.py filename.csv`. Is it clear where I'm coming from? https://stackoverflow.com/questions/66434805/how-to-pass-filename-to-vm-within-a-cloud-function The questions are related. – EB2127 Mar 02 '21 at 14:35
  • 1
    I updated the other question with the "startup script" option. – guillaume blaquiere Mar 02 '21 at 14:44