Auto Shutdown and Start Amazon EC2 Instance

Question

Can I automatically start and terminate my Amazon instance using Amazon API? Can you please describe how this can be done? I ideally need to start the instance and stop the instance at specified time intervals every day.

Answer 1

Just in case somebody stumbles on this ye old question, nowadays you can achieve the same thing by adding a schedule to an auto scaling group: increase the amount of instances in an auto scaling group to 1 at certain times and decrease it back to 0 afterwards.

And since this answer is getting a lot of views, I thought to link to a very helpful guide about this: Running EC2 Instances on a Recurring Schedule with Auto Scaling

Answer 2

You can try using the Amazon EC2 API tools directly. There are really only two commands you need: ec2-start-instances and ec2-stop-instances. Make sure that environment variables such as EC2_HOME, AWS_CREDENTIAL_FILE, EC2_CERT, EC2_PRIVATE_KEY, etc. are properly configured and all AWS credentials, certificate and private key files are in proper location - you can find more info in the AWS EC2 API tools documentation.

You can test the command by hand first and then, when everything works fine, configure Unix crontab or Scheduled Tasks on Windows. You can find the example below for the Linux /etc/crontab file (do not forget that all those environment variables mentioned above need to be present for 'your-account' user.

/etc/crontab
0 8     * * *   your-account ec2-start-instances <your_instance_id>
0 16    * * *   your-account ec2-stop-instances <your_instance_id>
# Your instance will be started at 8am and shutdown at 4pm.

I am a developer for the BitNami Cloud project, where we package the AWS tools (including the ones I mentioned) in a free, easy to use installer that you may want to try: BitNami CloudTools pack stack

Answer 3

I recommend you take a look at the EC2 Getting Started Guide, which shows you how to do what you need using the EC2 command line tools. You can easily script this into a cron job (on Linux / UNIX) or scheduled job on Windows to call the start and stop commands at a given time.

If you want to do this from your own code, you can use the SOAP or REST APIs; see the Developer Guide for details.

Answer 4

I wrote code in Python, using the Boto library, to do this. You can adjust this for your own use. Make sure to run this as part of a cron job, and then you will be able to start-up or shut-down as many instances as you need during the cron jobs run.

#!/usr/bin/python
#
# Auto-start and stop EC2 instances
#
import boto, datetime, sys
from time import gmtime, strftime, sleep

# AWS credentials
aws_key = "AKIAxxx"
aws_secret = "abcd"

# The instances that we want to auto-start/stop
instances = [
    # You can have tuples in this format:
    # [instance-id, name/description, startHour, stopHour, ipAddress]
    ["i-12345678", "Description", "00", "12", "1.2.3.4"]
]

# --------------------------------------------

# If its the weekend, then quit
# If you don't care about the weekend, remove these three 
# lines of code below.
weekday = datetime.datetime.today().weekday()
if (weekday == 5) or (weekday == 6):
    sys.exit()

# Connect to EC2
conn = boto.connect_ec2(aws_key, aws_secret)

# Get current hour
hh = strftime("%H", gmtime())

# For each instance
for (instance, description, start, stop, ip) in instances:
    # If this is the hour of starting it...
    if (hh == start):
        # Start the instance
        conn.start_instances(instance_ids=[instance])
        # Sleep for a few seconds to ensure starting
        sleep(10)
        # Associate the Elastic IP with instance
        if ip:
            conn.associate_address(instance, ip)
    # If this is the hour of stopping it...
    if (hh == stop):
        # Stop the instance
        conn.stop_instances(instance_ids=[instance])

Answer 5

If it's not mission critical - A simplistic thing to do is to schedule batch file to run 'SHUTDOWN' (windows) at 3am every day. Then at least you don't run the risk of accidentally leaving an unwanted instance running indefinitely.

Obviously this is only half the story!

Answer 6

The company I work for had customers regularly asking about this so we've written a freeware EC2 scheduling app available here:

http://blog.simple-help.com/2012/03/free-ec2-scheduler/

It works on Windows and Mac, lets you create multiple daily/weekly/monthly schedules and lets you use matching filters to include large numbers of instances easily or includes ones that you add in the future.

Answer 7

AWS Data Pipeline is working fine. https://aws.amazon.com/premiumsupport/knowledge-center/stop-start-ec2-instances/

If you wish exclude days from starting (e.g. weekend) add a ShellCommandPrecondition object.

In AWS Console/Data Pipeline, create a new pipeline. It's easyer to edit/import a definition (JSON)

    {
"objects": [
{
  "failureAndRerunMode": "CASCADE",
  "schedule": {
    "ref": "DefaultSchedule"
  },
  "resourceRole": "DataPipelineDefaultResourceRole",
  "role": "DataPipelineDefaultRole",
  "pipelineLogUri": "s3://MY_BUCKET/log/",
  "scheduleType": "cron",
  "name": "Default",
  "id": "Default"
},
{
  "name": "CliActivity",
  "id": "CliActivity",
  "runsOn": {
    "ref": "Ec2Instance"
  },
  "precondition": {
    "ref": "PreconditionDow"
  },
  "type": "ShellCommandActivity",
  "command": "(sudo yum -y update aws-cli) && (#{myAWSCLICmd})"
},
{
  "period": "1 days",
  "startDateTime": "2015-10-27T13:00:00",
  "name": "Every 1 day",
  "id": "DefaultSchedule",
  "type": "Schedule"
},
{
  "scriptUri": "s3://MY_BUCKET/script/dow.sh",
  "name": "DayOfWeekPrecondition",
  "id": "PreconditionDow",
  "type": "ShellCommandPrecondition"
},
{
  "instanceType": "t1.micro",
  "name": "Ec2Instance",
  "id": "Ec2Instance",
  "type": "Ec2Resource",
  "terminateAfter": "50 Minutes"
}
],
"parameters": [
{
  "watermark": "aws [options] <command> <subcommand> [parameters]",
  "description": "AWS CLI command",
  "id": "myAWSCLICmd",
  "type": "String"
}
 ],
"values": {
"myAWSCLICmd": "aws ec2 start-instances --instance-ids i-12345678 --region eu-west-1"
}
}

Put the Bash script to be dowloaded and executed as precondition in your S3 bucket

#!/bin/sh
if [ "$(date +%u)" -lt 6 ]
then exit 0
else exit 1
fi

On activating and running the pipeline on weekend days, the AWS console Pipeline Health Status reads a misleading "ERROR". The bash script returns an error (exit 1) and EC2 isn't started. On days 1 to 5, the status is "HEALTHY".

To stop EC2 automatically at closing office time, use the AWS CLI command daily wihtout precondition.

Answer 8

AutoScaling is limited to terminating instances. If you want to stop an instance and retain the server state then an external script is the best approach.

You can do this by running a job on another instance that is running 24/7 or you can use a 3rd party service such as Ylastic (mentioned above) or Rocket Peak.

For example in C# the code to stop a server is quite straightforward:

public void stopInstance(string instance_id, string AWSRegion)
        {
            RegionEndpoint myAWSRegion = RegionEndpoint.GetBySystemName(AWSRegion);
            AmazonEC2 ec2 = AWSClientFactory.CreateAmazonEC2Client(AWSAccessKey, AWSSecretKey, myAWSRegion);
            ec2.StopInstances(new StopInstancesRequest().WithInstanceId(instance_id));
        }

Answer 9

IMHO adding a schedule to an auto scaling group is the best "cloud like" approach as mentioned before.

But in case you can't terminate your instances and use new ones, for example if you have Elastic IPs associated with etc.

You could create a Ruby script to start and stop your instances based on a date time range.

#!/usr/bin/env ruby

# based on https://github.com/phstc/amazon_start_stop

require 'fog'
require 'tzinfo'

START_HOUR = 6 # Start 6AM
STOP_HOUR  = 0 # Stop  0AM (midnight)

conn = Fog::Compute::AWS.new(aws_access_key_id:     ENV['AWS_ACCESS_KEY_ID'],
                             aws_secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'])

server = conn.servers.get('instance-id')

tz = TZInfo::Timezone.get('America/Sao_Paulo')

now = tz.now

stopped_range = (now.hour >= STOP_HOUR && now.hour < START_HOUR)
running_range = !stopped_range

if stopped_range && server.state != 'stopped'
  server.stop
end

if running_range && server.state != 'running'
  server.start

  # if you need an Elastic IP
  # (everytime you stop an instance Amazon dissociates Elastic IPs)
  #
  # server.wait_for { state == 'running' }
  # conn.associate_address server.id, 127.0.0.0
end

Have a look at amazon_start_stop to create a scheduler for free using Heroku Scheduler.

Answer 10

Even though there are ways to achieve this using auto scaling, it might not suitable for all the occasions as it terminates the instances. Cron jobs will never work for a single instance (although it can perfectly be used for situations like stopping a single instance and scheduling other instances when running many instances). You can use API calls like StartInstancesRequest, and StopInstancesRequest to achieve the same but again you have to rely on a third resource. There are many applications to schedule AWS instances with many features but for a simple solution I would recommend a free app like snapleaf.io

Answer 11

You could look at Ylastic to do this. The alternative seems to be having one machine running that shuts down/starts other instances using a cron job or scheduled task.

Obviously if you only want one instance this is an expensive solution, as one machine has to always be running, and paying ~$80 a month for one machine to run cron jobs isn't cost effective.

Answer 12

Yes, you can do that using AWS Lambda. You can select the trigger in Cloudwatch which runs on Cron expressions on UTC.

Here is a related link https://aws.amazon.com/premiumsupport/knowledge-center/start-stop-lambda-cloudwatch/

Another alternative is to use awscli which is available from pip, apt-get, yum or brew, and then running aws configure with your credentials exported from IAM and executing the following bash script, to stop an EC2 that has been tagged with Name: Appname and Value: Appname Prod. You can use awscli to tag your instances or tag it manually from the AWS console. aws ec2 stop-instances will stop the instance and jq is used to filter the json query and fetch the correct instance id using the tags from aws ec2 describe-instances.

To verify that aws configure was successful and returns json output run aws ec2 describe-instances and your running instance id should be there in the output. Here is a sample output

{
    "Reservations": [
        {
            "Instances": [
                {
                    "Monitoring": {
                        "State": "disabled"
                    },
                    "PublicDnsName": "ec2-xxx.ap-south-1.compute.amazonaws.com",
                    "State": {
                        "Code": xx,
                        "Name": "running"
                    },
                    "EbsOptimized": false,
                    "LaunchTime": "20xx-xx-xxTxx:16:xx.000Z",
                    "PublicIpAddress": "xx.127.24.xxx",
                    "PrivateIpAddress": "xxx.31.3.xxx",
                    "ProductCodes": [],
                    "VpcId": "vpc-aaxxxxx",
                    "StateTransitionReason": "",
                    "InstanceId": "i-xxxxxxxx",
                    "ImageId": "ami-xxxxxxx",
                    "PrivateDnsName": "ip-xxxx.ap-south-1.compute.internal",
                    "KeyName": "node",
                    "SecurityGroups": [
                        {
                            "GroupName": "xxxxxx",
                            "GroupId": "sg-xxxx"
                        }
                    ],
                    "ClientToken": "",
                    "SubnetId": "subnet-xxxx",
                    "InstanceType": "t2.xxxxx",
                    "NetworkInterfaces": [
                        {
                            "Status": "in-use",
                            "MacAddress": "0x:xx:xx:xx:xx:xx",
                            "SourceDestCheck": true,
                            "VpcId": "vpc-xxxxxx",
                            "Description": "",
                            "NetworkInterfaceId": "eni-xxxx",
                            "PrivateIpAddresses": [
                                {
                                    "PrivateDnsName": "ip-xx.ap-south-1.compute.internal",
                                    "PrivateIpAddress": "xx.31.3.xxx",
                                    "Primary": true,
                                    "Association": {
                                        "PublicIp": "xx.127.24.xxx",
                                        "PublicDnsName": "ec2-xx.ap-south-1.compute.amazonaws.com",
                                        "IpOwnerId": "xxxxx"
                                    }
                                }
                            ],
                            "PrivateDnsName": "ip-xxx-31-3-xxx.ap-south-1.compute.internal",
                            "Attachment": {
                                "Status": "attached",
                                "DeviceIndex": 0,
                                "DeleteOnTermination": true,
                                "AttachmentId": "xxx",
                                "AttachTime": "20xx-xx-30Txx:16:xx.000Z"
                            },
                            "Groups": [
                                {
                                    "GroupName": "xxxx",
                                    "GroupId": "sg-xxxxx"
                                }
                            ],
                            "Ipv6Addresses": [],
                            "OwnerId": "xxxx",
                            "PrivateIpAddress": "xx.xx.xx.xxx",
                            "SubnetId": "subnet-xx",
                            "Association": {
                                "PublicIp": "xx.xx.xx.xxx",
                                "PublicDnsName": "ec2-xx.ap-south-1.compute.amazonaws.com",
                                "IpOwnerId": "xxxx"
                            }
                        }
                    ],
                    "SourceDestCheck": true,
                    "Placement": {
                        "Tenancy": "default",
                        "GroupName": "",
                        "AvailabilityZone": "xx"
                    },
                    "Hypervisor": "xxx",
                    "BlockDeviceMappings": [
                        {
                            "DeviceName": "/dev/xxx",
                            "Ebs": {
                                "Status": "attached",
                                "DeleteOnTermination": true,
                                "VolumeId": "vol-xxx",
                                "AttachTime": "20xxx-xx-xxTxx:16:xx.000Z"
                            }
                        }
                    ],
                    "Architecture": "x86_64",
                    "RootDeviceType": "ebs",
                    "RootDeviceName": "/dev/xxx",
                    "VirtualizationType": "xxx",
                    "Tags": [
                        {
                            "Value": "xxxx centxx",
                            "Key": "Name"
                        }
                    ],
                    "AmiLaunchIndex": 0
                }
            ],
            "ReservationId": "r-xxxx",
            "Groups": [],
            "OwnerId": "xxxxx"
        }
    ]
}

The following bash script is stop-ec2.sh in /home/centos/cron-scripts/ which is inspired from this SO post

(instance=$(aws ec2 describe-instances | jq '.Reservations[].Instances | select(.[].Tags[].Value | startswith("Appname Prod") ) |  select(.[].Tags[].Key == "Appname") |  {InstanceId: .[].InstanceId, PublicDnsName: .[].PublicDnsName, State: .[].State, LaunchTime: .[].LaunchTime, Tags: .[].Tags}  | [.]' | jq -r .[].InstanceId) && aws ec2 stop-instances --instance-ids ${instance} )

Run the file using sh /home/centos/cron-scripts/stop-ec2.sh and verify that the EC2 instance gets stopped. To debug run aws ec2 describe-instances | jq '.Reservations[].Instances | select(.[].Tags[].Value | startswith("Appname Prod") ) | select(.[].Tags[].Key == "Appname") | {InstanceId: .[].InstanceId, PublicDnsName: .[].PublicDnsName, State: .[].State, LaunchTime: .[].LaunchTime, Tags: .[].Tags} | [.]' | jq -r .[].InstanceId and see that it returns the correct instance ID which has been tagged.

Then in crontab -e the following line can be added

30 14 * * * sh /home/centos/cron-scripts/stop-ec2.sh >> /tmp/stop

which will log the output to /tmp/stop. The 30 14 * * * is the UTC cron expression that you can check in https://crontab.guru/. Similarly replacing with aws ec2 start-instances can start an instance.

Answer 13

I believe that the initial question was a little bit confusing. It depends on what Pasta needs: 1.launch/terminate (instance store) - Auto Scaling is the right solution( Nakedible's answer) 2.start/stop EBS boot instance - Auto Scaling won't help, I use remote scheduled scripts (i.e., ec2 CLI).

Answer 14

You cannot do this automatically, or at least not without some programming and API manipulation in script files. If you want to a reliable solution to stop, restart and manage your images (presumably to control costs in your environment) then you may want to look at LabSlice. Disclaimer: I work for this company.