How to pass commands to ec2 after starting an instance from python script?

Question

I am trying to run a python script that can start an instance and then run some commands. I know it can be passed if I try to create an instance in the form of userdata. What I am trying to figure out is how to pass it if I am starting an already created instance. Following is the code using boto3 to pass simple hello world while creating an instance:

import boto3
userdata = """#cloud-config

repo_update: true
repo_upgrade: all

packages:
 - s3cmd

runcmd:
 - echo 'Hello S3!' > /tmp/hello.txt
 - aws --region YOUR_REGION s3 cp /tmp/hello.txt s3://YOUR_BUCKET/hello.txt
"""

ec2 = boto3.resource('ec2')
instances = ec2.create_instances(
    ImageId='ami-f5f41398',         # default Amazon linux
    InstanceType='t2.micro',
    KeyName='YOUR_SSH_KEY_NAME',
    MinCount=1,
    MaxCount=1,
    IamInstanceProfile={
        'Arn': 'YOUR_ARN_ID'
    },
    SecurityGroupIds=['YOUR_SECURITY_GROUP_NAME'],
    UserData=userdata
)

Something like

i = ec2.Instance(id='i-5fea4d42')
i.start('pass commands here: eg echo xx, mv a b/ etc')

You want to run different command every time or the same command you have in user-data? — helloV, Nov 17 '16 at 14:50
I am trying to execute commands on the ec2 shel. They will be different, I am trying to use boto and command shell, which was not carried over to boto3 link. Probably I will use a combination of paramiko and boto3. — Krishna Aswani, Nov 17 '16 at 19:55

score 2 · Accepted Answer · answered Jan 08 '17 at 23:15

I was able to use paramiko along with boto3 and awscli to run everything on cloud in an automated way.

import paramiko
import boto3
import time
import subprocess
import awscli
import os

# Starting instance, copying data to instance, running model, copying output to s3
print ("\nCreating ssh session")
session = boto3.Session()
ec2 = session.resource('ec2', region_name='us-east-1')
i = ec2.Instance(id='instance id') # instance id


print('\nstarting deeplearning_ami instance')
i.start()
i.wait_until_running()
i.load()
print ("Waiting for the checks to finish..")
time.sleep(45)

k = paramiko.RSAKey.from_private_key_file(key_path) #your private key for ssh
c = paramiko.SSHClient()
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
print ("\nConnecting to shell using ssh")
c.connect( hostname = i.public_dns_name, username = "ec2-user", pkey = k )

print ("\nExecuting commands on EC2 instance\n")

stdin , stdout, stderr = c.exec_command(""" mkdir -p model;
                                        printf 'Creating directory structure, if it does not exists \n';
                                        cd model;
                                        mkdir -p data out cp;
                                        printf '\nCopying code from s3 bucket to ec2 instance \n';
                                        aws s3 cp {0} .;
                                        printf '\nDownloading data from s3 bucket to ec2 instance';
                                        aws s3 sync {1} data;
                                        printf '\n Download complete, running model..\n\n';
                                        python theano_test.py;
                                        echo 'Hello S3!' > out/hello.txt;
                                        printf '\n Copying output to s3 bucket \n';
                                        aws s3 sync out {2} """.format(s3_code,s3_data,s3_out), get_pty=True)

for line in iter(lambda: stdout.readline(2048), ""): print(line, end="")
print ("\n \n Processingfinished")

#print ('Stopping instance')
i.stop() 


print ('Copying output from s3 bucket to local')
os.system('aws s3 sync ' + s3_out + ' ' + local_out)

score 0 · Answer 2 · edited May 23 '17 at 12:07

0

You can set the UserData and start the instance and it will pick up the change: http://boto3.readthedocs.io/en/latest/reference/services/ec2.html#EC2.Client.modify_instance_attribute

However, UserData scripts are usually only set to run once, so you will have to modify it to run on every boot: How do I make cloud-init startup scripts run every time my EC2 instance boots?

Another way would be to set up SSH and run a script after boot, but shoving this in UserData is cleaner in my opinion.

edited May 23 '17 at 12:07

Community

1
1

answered Nov 17 '16 at 15:40

louahola

2,088
1
15
12

I am trying to use boto and command shell, which was not carried over to boto3 [link](http://stackoverflow.com/questions/34028219/how-to-execute-commands-on-aws-instance-using-boto3). Probably I will use a combination of paramiko and boto3. – Krishna Aswani Nov 17 '16 at 19:53

How to pass commands to ec2 after starting an instance from python script?

2 Answers2