Can I replicate `create-with-container` gcloud sdk command using the googleapiclient python client

Question

I currently have a docker container on the gcloud container repository which I want to launch an instance of in order to run a calculation, save a result and close:

gcloud compute instances create-with-container test-1 \
--machine-type=n1-standard-4 \
--boot-disk-size=20GB \
--container-image=eu.gcr.io/<container-link> \
--container-env=GCLOUD_INPUT_FILENAME=file.txt \
--container-env=GCLOUD_PROJECT=project-name

However, I want to be able to launch these instances using a web-interface (flask) which implies I want to use the googleapiclient (python) in order to create and manage these instances:

It looks like while you can create a instance creation order using the discovery api:

compute = googleclientapi.discovery.build('compute', 'v1')
compute.instances().insert(...).execute()

but it doesn't look like it is possible to emulate create-with-container gcloud sdk command, although you can pass 'machineImage' as part of the creation request.

Can one create a compute instance 'with-container' without using subprocess to call the gcloud sdk

OR

Can I convert my create-with-container instance into a machine image and then use the googleapi client?

Try running the gcloud command with `--log-http` enabled. You can see what google API call it makes and copy it. — Hitobat, Sep 13 '20 at 19:27
It's possible that several different calls are performed by GCLOUD. the log http can show you all the call to implement by yourselves to achieve the same behavior — guillaume blaquiere, Sep 14 '20 at 08:12
Was not able to find a good answer using `--log-http`. Would love to know. Commenting to follow. — David Bernat, Oct 16 '20 at 04:36
The compute instance REST request contained yaml file serialised within a dictionary. Behind the scenes, a standard containerisedOS (cos) container is started while the container which should be run on-top was passed via dictionary which itself container the yaml file. I am using cloudlib as the driver for creating compute instances - I can just pass the serialized yaml as a dictionary using the ex_metadata parameter. yaml payload dictionary: `{"items": [ "gce-container-declaration", "value": ""}, "google-logging-enabled", "value": "true"}]}` — M. Williams, Nov 13 '20 at 18:40
IIRC this is achieved by `gcloud` with a single REST call ([`instances.insert`](https://cloud.google.com/compute/docs/reference/rest/v1/instances/insert)). The "trick" per @m-williams is to populate the metadata correctly with the container (cloud?) init. I'm headed out for the day but, if this is still open later and I have time, I'll try to draft a solution. I would recommend using @hitobat suggestion to `--log-http` and check the POST body too. Or use Cloud Console to construct an exemplar `create-with-container` and then "Equivalent REST" to show the structure you'll need for the body. — DazWilkin, Oct 22 '22 at 01:42

score 0 · Answer 1 · answered Mar 02 '21 at 09:46

Just for FYI, what I found from the log-http logging(I know it is not the right answer)

==== request start ====
uri: https://compute.googleapis.com/batch/compute/v1
method: POST
== headers start ==
b'authorization': --- Token Redacted ---
b'content-length': b'479'
b'content-type': b'multipart/mixed; boundary="===============34234234234234=="'
b'user-agent': b'google-cloud-sdk gcloud/329.0.0 command/gcloud.compute.instances.create-with-container invocation-id/0dd6a37ac0624ac4b00e30a44da environment/devshell environment-version/None interactive/False from-script/False python/3.7.3 term/screen (Linux 5.4.89+)'
== headers end ==

"create-with-container" is passed through the header, I did not find the body for the same.

DazWilkin · Answer 2 · 2022-10-22T21:42:33.153

The code would benefit from refinement but here's a solution:

import os

from google.cloud import compute_v1
from google.cloud.compute_v1 import types


project=os.getenv("PROJECT")
number=os.getenv("NUMBER")
zone=os.getenv("ZONE")

machine_type=os.getenv("TYPE")

name=os.getenv("NAME")

# Compute Engine default
email=f"{number}-compute@developer.gserviceaccount.com"

# Define a non-trivial container with command and args
value="""
spec:
  containers:
  - name: foo
    image: gcr.io/google-containers/busybox@sha256:d8d3bc2c183ed2f9f10e7258f84971202325ee6011ba137112e01e30f206de67
    command:
    - /bin/sh
    args:
    - -c
    - |
      while true
      do
        echo "Hello"
        sleep 15
      done
    stdin: true
    tty: true
    restartPolicy: Always
"""

client = compute_v1.InstancesClient()

request = types.InsertInstanceRequest(
    project=project,
    zone=zone,
    instance_resource=types.Instance(
        name=name,
        machine_type=f"zones/{zone}/machineTypes/{machine_type}",
        disks=[
            types.AttachedDisk(
                device_name="foo",
                boot=True,
                auto_delete=True,
                initialize_params=types.AttachedDiskInitializeParams(
                    disk_size_gb=10,
                    disk_type=f"zones/{zone}/diskTypes/pd-balanced",
                    source_image="projects/cos-cloud/global/images/cos-stable-101-17162-40-13",
                ),
            ),
        ],
        network_interfaces=[
            types.NetworkInterface(
                access_configs=[
                    types.AccessConfig(
                        name="External NAT",
                        network_tier="STANDARD",
                    ),
                ],
            ),
        ],
        metadata=types.Metadata(
            items=[{
                "key":"gce-container-declaration",
                "value": value,
            }],
        ),
        service_accounts=[
            types.ServiceAccount(
                email=email,
                scopes=[
                    "https://www.googleapis.com/auth/devstorage.read_only",
                    "https://www.googleapis.com/auth/logging.write",
                    "https://www.googleapis.com/auth/monitoring.write",
                    "https://www.googleapis.com/auth/servicecontrol",
                    "https://www.googleapis.com/auth/service.management.readonly",
                    "https://www.googleapis.com/auth/trace.append"
                ],
            ),
        ],
    ),
)

response=client.insert(
    request=request,
)

Can I replicate `create-with-container` gcloud sdk command using the googleapiclient python client

2 Answers2