How to configure vpc/subnets for ec2 instances via cml runner in aws?

Question

I am setting up a workflow via dvc(https://dvc.org/) and CML (https://cml.dev/) in aws. I am gitlab user. based on documentation provided in CML, cml runner can provision ec2 instances in aws and run training and processing job. how can i pass vpc/subnet configuration for ec2 via cml runner.

deploy:
  image: iterativeai/cml:0-dvc2-base1
  script:
    - cml runner launch
      --cloud=gcp
      --cloud-region=asia-southeast1-a
      ...
      --labels=cml-runner

test:
  needs: [deploy]
  script:
    - ...

This comment / issue might be relevant for you https://github.com/iterative/cml/issues/826#issuecomment-986839509 — Shcheklein, Mar 19 '23 at 22:29

score 2 · Accepted Answer · answered Mar 20 '23 at 19:42

2

Here is an example that you can use:

cml runner launch \
   ...
   --cloud=aws \
   --cloud-region=xxx \
   --cloud-type=t3.small \
   --cloud-aws-security-group=cml-test-sg \
   --cloud-aws-subnet=subnet-5d7ee225

You can create an AWS security group in the VPC you want to target (you need to expose SSH for cml runner) note that cml-test-sg is the Name of the created security in your target VPC

Additionally you can target a specific subnet within your VPC by also providing the id for the resource you want, here: subnet-5d7ee225

answered Mar 20 '23 at 19:42

dabarnes

173
1
5

1

I'll add that in your example you are using GCP and not AWS, there currently no equivalent option when using GCP. (I would create an issue on GitHub if you need that) – dabarnes Mar 20 '23 at 19:45
thanks. can you add any specific tags to this as well? – kyagu Mar 20 '23 at 21:29
1

you can use `--cloud-metadata="SomeKey=SomeValue"` – dabarnes Mar 21 '23 at 04:12

How to configure vpc/subnets for ec2 instances via cml runner in aws?

1 Answers1