18

I'd like to launch an EC2 Windows instance, upload an EXEecutable & execute it (all in an automated fashion, this is important)

So far I was able to programmatically launch EC2 Windows instance & get its parameters (password / IP), now I'd like to find a way to upload this Executable (from my Windows machine or from my other EC2 linux instance) & run it.

I thought about launching an RDP connection & using a macro software to upload & execute the file, but based on previous experiences this is a poor/fragile approach to say the least.

I also thought about uploading this EXE to a server, then do something like this on Windows:

wget http://www.domain.com/my-file.exe

Except that Windows doesn't have wget!

So my question is: is there a way to programmatically upload & execute an EXEcutable in EC2 Windows instance?

gareth_bowles
  • 20,760
  • 5
  • 52
  • 82
style-sheets
  • 727
  • 3
  • 7
  • 15

5 Answers5

12

An alternative approach is to use Windows powershell and WinRM - it allows for remote execution, a bit like ssh on Linux.

Here is a sample of a powershell script you can run on the client to remote execute a script (taken from: https://github.com/CloudifySource/cloudify/blob/master/esc/src/main/resources/clouds/ec2-win/upload/bootstrap-client.ps1):

param ([string]$target, [string]$username, [string]$password, [string]$command)

$ErrorActionPreference="Stop"

# Set up the password
$securePassword = ConvertTo-SecureString -AsPlainText -Force $password
$cred = New-Object System.Management.Automation.PSCredential $username, $securePassword

Write-Host "Connecting to management service of $target"
Connect-WSMan -Credential $cred $target 

set-item WSMan:\$target\Client\TrustedHosts -Value * -Force
set-item WSMan:\$target\Shell\MaxMemoryPerShellMB -Value 0 -Force

Write-Host Invoking command on Remote host $target
Invoke-Command -ComputerName $target -Credential $cred  -ScriptBlock {  
    Invoke-Expression $args[0]
} -ArgumentList $command
Write-Host "Command finished"

You can run this command from your own script with the following command:

powershell.exe -inputformat none -File PATH_TO_SCRIPT -target TARGET_IP -password PASSWORD -username USERNAME -command COMMAND_TO_EXECUTE

You should probably quote your strings, especially the password and command, as these will usually have special characters that powershell can interpret as something else.

The WinRM service is on by default on the EC2 Amazon Windows AMIs. All you need to do is open port 5985 (the WinRM port) in your security group.

Finally, if you have never used powershell remoting on your client machine before, there are a couple of commands you should execute to set it up (you only need to do this once):

set-item WSMan:\localhost\Client\TrustedHosts -Value * -Force
set-item WSMan:\localhost\Shell\MaxMemoryPerShellMB -Value 0 -Force
Enable-PSRemoting
Set-ExecutionPolicy unrestricted

Make sure to run these as an Administrator.

Eliran Malka
  • 15,821
  • 6
  • 77
  • 100
Barak
  • 3,066
  • 2
  • 20
  • 33
7

The command ec2-run-instances has two additional arguments that can be used when running the command. The user-data command and user-data-file both of these perform the same task just they read from different input. When you use this argument the contents of the user-data will be uploaded to a amazon hosted URI http://169.254.169.254/1.0/user-data only available to the instance that was launched.

The normal way to do this in the linux environment would be to upload a shell script to the instance to download the exe, your user-data-file might look something like this...

#! /bin/bash
wget http://www.domain.com/my-file.exe

In Windows there's no default service installed to execute the user-data-file when the instance is booted but there is an open-source project CloudInit.NET which simulates the same process but with a powershell script. The only requirements are .NET 4.0 and CloudInit.NET. Once installed it will execute the user-data-file when the instance is booted. It's very easy to download a file and execute it with a powershell script.

!# /powershell/
$wc = New-Object System.Net.WebClient
$wc.DownloadFile("http://www.domain.com/my-file.exe", "C:\my-file.exe");
& 'C:\my-file.exe'
bwight
  • 3,300
  • 17
  • 21
  • Thank you, but as I said to Chakri/in my original question: I want 100% automated access. If I understood you correctly, I must manually install the service every time I launch the instance (new instances are launched programmatically, following your advice means that I have to manually connect every time)...am I missing something here? – style-sheets Apr 20 '12 at 21:10
  • 2
    No you launch one instance, install the service, then save the instance as an AMI and when you launch future instance you use the AMI that already has the service installed. – bwight Apr 20 '12 at 22:26
  • Sorry for the late reply...had to test thoroughly! Your last comment (about making AMIs) was the **real** answer for me, thank you very much! – style-sheets Apr 27 '12 at 12:08
1

You can handle this in 2 ways,

  • Using winscp in Windows SFTP program.

    To access your Amazon server using SFTP on Windows, download a Windows SFTP application. Using WinSCP you’ll establish an SFTP session with your server. WinSCP offers some nice features that make it easy to work with your EC2 server. For example, a command in the button bar opens a PuTTY SSH terminal session using the same credentials you used for your SFTP session. (You can also launch a PuTTY session by clicking CTRL+P.).

  • Get an S3 bucket and mount on all your windows and linux EC2 instances. You should be able to upload and download the files to S3 bucket from your workstation, which are accessible to your instances.

Chakri
  • 173
  • 6
  • As I said: I want 100% automated access. Both methods requires manual intervention (new instances are launched programmatically, following your advice means that I have to manually connect every time)...am I missing something here? – style-sheets Apr 19 '12 at 22:19
  • I was just mentioning the core method of connectivity. Here is more detailed steps. Option a> You write a batch script that takes your file & server_name as input. Then put it in your cron/task-manager as shown c:\scripts\upload.bat file_name server_name Option b> You map the S3 drive to both your client and server. Create a script that put files to the mapped drive. c:\scripts\upload.bat mapped_drive – Chakri Apr 20 '12 at 00:47
  • Oh I see. Nice idea, this should solve the upload part, but I still can't execute the EXE once it's uploaded to the Windows instance, can I? – style-sheets Apr 20 '12 at 10:54
  • The Windows SFTP provides windows ssh command. You can use that for any remote execution. Just like how you do it linux with ssh key exchange. – Chakri Apr 20 '12 at 14:02
  • Thank you Chakri, can you please expand on the SFTP connection? I researched it a lot, but it seems that, beside the SSH port opening in the EC2's security groups, it's not available out of the box (ie. I have to manually install *something* to get the SFTP to work) – style-sheets Apr 20 '12 at 21:19
  • Little confused which sftp connection question, You mean how to execute command using ssh or connecting to windows"EC2" sftp ? – Chakri Apr 22 '12 at 20:23
1

This sounds like a perfect use case of CloudFormation. I created a template that demonstrates. To use, put your executable in an S3 bucket and create a new CloudFormation stack with the following template. It will download your executable from S3 and run it. Note: the template makes use of special AMIs with CloudFormationScripts built in.

{
    "AWSTemplateFormatVersion" : "2010-09-09",
    "Description" : "Spot Autoscaling for installing and running Windows Services.",
    "Parameters" : {
        "InstanceType" : {
            "Description" : "WebServer EC2 instance type",
            "Type" : "String",
            "Default" : "m1.small",
            "AllowedValues" : ["t1.micro", "m1.small", "m1.medium", "m1.large", "m1.xlarge", "m2.xlarge", "m2.2xlarge", "m2.4xlarge", "c1.medium", "c1.xlarge", "cc1.4xlarge", "cc2.8xlarge", "cg1.4xlarge"],
            "ConstraintDescription" : "must be a valid EC2 instance type."
        },
        "KeyName" : {
            "Description" : "The EC2 Key Pair to get Admin password to Instance.",
            "Type" : "String"
        },
        "DeployS3Bucket" : {
            "Description" : "The S3 Bucket where deploy files are stored",
            "Type" : "String"
        },
        "DeployS3Key" : {
            "Description" : "The exe file that runs on startup",
            "Type" : "String"
        }
    },
    "Mappings" : {
        "RegionToAMIMap" : {
            "us-east-1" : {
                "AMI" : "ami-60b90609"
            },
            "us-west-1" : {
                "AMI" : "ami-5bd6f11e"
            },
            "eu-west-1" : {
                "AMI" : "ami-07151573"
            },
            "ap-southeast-1" : {
                "AMI" : "ami-6ab5f538"
            },
            "ap-northeast-1" : {
                "AMI" : "ami-424ff043"
            }
        }
    },
    "Resources" : {
        "IAMUser" : {
            "Type" : "AWS::IAM::User",
            "Properties" : {
                "Path" : "/",
                "Policies" : [{
                    "PolicyName" : "root",
                    "PolicyDocument" : {
                        "Statement" : [{
                            "Effect" : "Allow",
                            "Action" : "*",
                            "Resource" : "*"
                        }]
                    }
                }]
            }
        },
        "IAMUserAccessKey" : {
            "Type" : "AWS::IAM::AccessKey",
            "Properties" : {
                "UserName" : {
                    "Ref" : "IAMUser"
                }
            }
        },
        "SecurityGroup" : {
            "Type" : "AWS::EC2::SecurityGroup",
            "Properties" : {
                "GroupDescription" : "Enable RDP",
                "SecurityGroupIngress" : [{
                    "IpProtocol" : "tcp",
                    "FromPort" : "3389",
                    "ToPort" : "3389",
                    "CidrIp" : "0.0.0.0/0"
                }]
            }
        },
        "RunExecutable" : {
            "Type" : "AWS::EC2::Instance",
            "Metadata" : {
                "AWS::CloudFormation::Init" : {
                    "config" : {
                        "files" : {
                            "c:\\ToRun\\executable.exe" : {
                                "source" : {
                                    "Fn::Join" : ["/", ["http://s3.amazonaws.com", {
                                        "Ref" : "DeployS3Bucket"
                                    }, {
                                        "Ref" : "DeployS3Key"
                                    }]]
                                },
                                "authentication" : "S3AccessCreds"
                            }
                        },
                        "commands" : {
                            "1-run-executable" : {
                                "command" : "c:\\ToRun\\executable.exe"
                            }
                        }
                    }
                },
                "AWS::CloudFormation::Authentication" : {
                    "S3AccessCreds" : {
                        "type" : "S3",
                        "accessKeyId" : {
                            "Ref" : "IAMUserAccessKey"
                        },
                        "secretKey" : {
                            "Fn::GetAtt" : ["IAMUserAccessKey", "SecretAccessKey"]
                        },
                        "buckets" : [{
                            "Ref" : "DeployS3Bucket"
                        }]
                    }
                }
            },
            "Properties" : {
                "KeyName" : {
                    "Ref" : "KeyName"
                },
                "ImageId" : {
                    "Fn::FindInMap" : ["RegionToAMIMap", {
                        "Ref" : "AWS::Region"
                    }, "AMI"]
                },
                "SecurityGroups" : [{
                    "Ref" : "SecurityGroup"
                }],
                "InstanceType" : {
                    "Ref" : "InstanceType"
                },
                "UserData" : {
                    "Fn::Base64" : {
                        "Fn::Join" : ["", ["<script>\n", "cfn-init.exe -v -s ", {
                            "Ref" : "AWS::StackName"
                        }, " -r RunExecutable ", " --access-key ", {
                            "Ref" : "IAMUserAccessKey"
                        }, " --secret-key ", {
                            "Fn::GetAtt" : ["IAMUserAccessKey", "SecretAccessKey"]
                        }, "\n", "</script>"]]
                    }
                }
            }
        }
    },
    "Outputs" : {}
}
Eliran Malka
  • 15,821
  • 6
  • 77
  • 100
Edwin
  • 1,458
  • 10
  • 17
0

I had do a similar automation for AWS for enterprise deployment back in 2011. Amazon cloud formation and opsworks where still under construction then.However we had successfully done deployment automation using dotnet for both linux and windows platform.The powershell and ftp model where ruled out becasue it was an enterprise environment and there was port restriction. Below are the approaches i used.

Note:This was asp.net web application

For linux deployment.

We used an open source project called sharpshell(sharpSSH).This is c# application which simpulates shell communication between windows and linux.Just need to supply the target aws address and security key to connect. We customized the app around our requirement

For windows

Having said that cloud formation API where still not available the and less documentation available by AWS. We used a workaround, webservice approach. Crearted a webservice which basically uploads a file to server and deploy. With this webserver hosted on amazaon windows server.Created a base image out of this and certificate. Finally a new instances created out of this image would have webservices hosted which could be called to upload the deployment package and install on that system.

Though the above startergies where not fool proof and had less controll. we achieved the functionality of cross platform direct deployment from windows to linux without using S3 buckets or powershell.

Please let me know if you require any clarification.

Cheers! Charles

charles anil dsouza
  • 1