Cloudformation template order of execution?

Question

I am trying to create a cloudformation template which installs puppet and also aws puppet module. I am able to create my instance with puppet, define the security group etc. and it seems to work fine, but I also want to install the aws puppet module as part of my template. This is the code for my puppet instance

    "PuppetMasterInstance" : {
  "Type" : "AWS::EC2::Instance",
  "Metadata" : {
    "AWS::CloudFormation::Init" : {
      "config" : {
        "packages" : {
          "yum" : {
          "puppet3" : [],
            "puppet3-server" : [],
            "ruby-devel" : [],
            "gcc" : [],
            "make" : [],
            "rubygems" : []
          },
          "rubygems" : {
            "json" : []
          }
        },
        "files": {
          "/etc/yum.repos.d/epel.repo": {
            "source": "https://s3.amazonaws.com/cloudformation-examples/enable-epel-on-amazon-linux-ami",
            "mode": "000644",
            "owner": "root",
            "group": "root"
          },

          "/etc/puppet/autosign.conf": {
            "content": "*.internal\n",
            "mode": "100644",
            "owner": "root",
            "group": "wheel"
          },
          "/etc/puppet/fileserver.conf": {
            "content": "[modules]\n allow *.internal\n",
            "mode": "100644",
            "owner": "root",
            "group": "wheel"
          },
          "/etc/puppet/puppet.conf": {
            "content": {
              "Fn::Join": [
                "",
                [
                  "[main]\n",
                  " logdir=/var/log/puppet\n",
                  " rundir=/var/run/puppet\n",
                  " ssldir=$vardir/ssl\n",
                  " pluginsync=true\n",
                  "[agent]\n",
                  " classfile=$vardir/classes.txt\n",
                  " localconfig=$vardir/localconfig\n"
                ]
              ]
            },
            "mode": "000644",
            "owner": "root",
            "group": "root"
          },
          "/etc/puppet/modules/cfn/manifests/init.pp": {
            "content": "class cfn {}",
            "mode": "100644",
            "owner": "root",
            "group": "wheel"
          },
          "/etc/puppet/modules/cfn/lib/facter/cfn.rb": {
            "source": "https://s3.amazonaws.com/cloudformation-examples/cfn-facter-plugin.rb",
            "mode": "100644",
            "owner": "root",
            "group": "wheel"
          }
        },
        "services": {
          "sysvinit": {
            "puppetmaster": {
              "enabled": "true",
              "ensureRunning": "true"
            }
          }
        }
      }
    }
  },
  "Properties": {
    "InstanceType": {
      "Ref": "InstanceType"
    },
    "SecurityGroups": [
      {
        "Ref": "PuppetGroup"
      }
    ],
    "ImageId": {

      "Ref": "AmiID"

    },
    "KeyName": {
      "Ref": "KeyName"
    },
    "UserData": {
      "Fn::Base64": {
        "Fn::Join": [
          "",
          [
            "#!/bin/bash\n",
            "yum update -y \n",


            "/opt/aws/bin/cfn-init --region ",
            {
              "Ref": "AWS::Region"
            },
            " -s ",
            {
              "Ref": "AWS::StackName"
            },
            " -r PuppetMasterInstance ",
            " --access-key ",
            {
              "Ref": "CFNKeys"
            },
            " --secret-key ",
            {
              "Fn::GetAtt": [
                "CFNKeys",
                "SecretAccessKey"
              ]
            },
            "\n",
            "/opt/aws/bin/cfn-signal -e $? '",
            {
              "Ref": "PuppetMasterWaitHandle"
            },
            "'\n"
          ]
        ]
      }
    }
  }
}

This works fine, however I want to execute the following commands after puppet is installed:

"gem install aws-sdk-core",
"gem install retries",
"export AWS_ACCESS_KEY_ID=my_key",
"export AWS_SECRET_ACCESS_KEY=my_secret",
 "puppet module install puppetlabs-aws"

I tried using the "commands:" tag before "files:" and the template failed. I tried to put the code inside the "UserData": but it failed again. I couldn't find information about the order of execution of the different sections in the template and I assume the failures are due to wrong order of execution (puppet & ruby are not installed when the commands run).

Any help will be much appreciated.

I don't have time to write an example out of this but my first though it to write a bit of shell script that explictly waits for the install. The order of execution is for cloud-init, see this answer http://stackoverflow.com/questions/34095839/cloud-init-what-is-the-execution-order-of-cloud-config-directives — Vorsprung, Sep 05 '16 at 12:22
Thanks for the suggestion. I need to have everything inside my template and the only way I can think of to check if the services are installed is to write infinite loop where I check if the service is running, however I would imagine this will block the execution of the template and the whole thing will just end up waiting an infinite loop to finish which in turn is waiting for a service to get installed break out of the loop...There must be a proper way to execute my script after the instance packages are installed... — tintin, Sep 05 '16 at 13:04

score 3 · Accepted Answer · answered Sep 05 '16 at 15:10

I found a very informative post on the aws forum regarding the order of execution. AWS::CloudFormation::Init executes in the following order:

packages -> groups -> users-> sources -> files -> commands -> services

source: https://forums.aws.amazon.com/message.jspa?messageID=414670

The way I fixed the problem is probably far from ideal but it works:

    "PuppetMasterInstance" : {
  "Type" : "AWS::EC2::Instance",
  "Metadata" : {
    "AWS::CloudFormation::Init" : {
      "configSets" : {
        "ascending" : [ "config" , "config2" ],
        "descending" : [ "config2" , "config" ]
      },
      "config" : {
        "packages" : {
          "yum" : {
          "puppet3" : [],
            "puppet3-server" : [],
            "ruby-devel" : [],
            "gcc" : [],
            "make" : [],
            "rubygems" : []
          },
          "rubygems" : {
            "json" : []
          }
        },
        "files": {
          "/etc/yum.repos.d/epel.repo": {
            "source": "https://s3.amazonaws.com/cloudformation-examples/enable-epel-on-amazon-linux-ami",
            "mode": "000644",
            "owner": "root",
            "group": "root"
          },

          "/etc/puppet/autosign.conf": {
            "content": "*.internal\n",
            "mode": "100644",
            "owner": "root",
            "group": "wheel"
          },
          "/etc/puppet/fileserver.conf": {
            "content": "[modules]\n allow *.internal\n",
            "mode": "100644",
            "owner": "root",
            "group": "wheel"
          },
          "/etc/puppet/puppet.conf": {
            "content": {
              "Fn::Join": [
                "",
                [
                  "[main]\n",
                  " logdir=/var/log/puppet\n",
                  " rundir=/var/run/puppet\n",
                  " ssldir=$vardir/ssl\n",
                  " pluginsync=true\n",
                  "[agent]\n",
                  " classfile=$vardir/classes.txt\n",
                  " localconfig=$vardir/localconfig\n"
                ]
              ]
            },
            "mode": "000644",
            "owner": "root",
            "group": "root"
          },
          "/etc/puppet/modules/cfn/manifests/init.pp": {
            "content": "class cfn {}",
            "mode": "100644",
            "owner": "root",
            "group": "wheel"
          },
          "/etc/puppet/modules/cfn/lib/facter/cfn.rb": {
            "source": "https://s3.amazonaws.com/cloudformation-examples/cfn-facter-plugin.rb",
            "mode": "100644",
            "owner": "root",
            "group": "wheel"
          }
        },
        "services": {
          "sysvinit": {
            "puppetmaster": {
              "enabled": "true",
              "ensureRunning": "true"
            }
          }
        }
      },

      "config2" : {
        "commands" : {
          "1" : {
            "command" : "gem install aws-sdk-core"
          },
          "2" : {
            "command" : "gem install retries"
          },
          "3" : {
            "command" : "export _MYAWSKEY_"
          },
          "4" : {
            "command" : "export MY_AWS_SECRET_"
          },
          "5" : {
            "command" : "puppet module install puppetlabs-aws"
          }
        }

      }



    }
  },
  "Properties": {
    "InstanceType": {
      "Ref": "InstanceType"
    },
    "SecurityGroups": [
      {
        "Ref": "PuppetGroup"
      }
    ],
    "ImageId": {

      "Ref": "AmiID"

    },
    "KeyName": {
      "Ref": "KeyName"
    },
    "UserData": {
      "Fn::Base64": {
        "Fn::Join": [
          "",
          [
            "#!/bin/bash\n",
            "yum update -y \n",


            "/opt/aws/bin/cfn-init -c ascending --region ",
            {
              "Ref": "AWS::Region"
            },
            " -s ",
            {
              "Ref": "AWS::StackName"
            },
            " -r PuppetMasterInstance ",
            " --access-key ",
            {
              "Ref": "CFNKeys"
            },
            " --secret-key ",
            {
              "Fn::GetAtt": [
                "CFNKeys",
                "SecretAccessKey"
              ]
            },
            "\n",
            "/opt/aws/bin/cfn-signal -e $? '",
            {
              "Ref": "PuppetMasterWaitHandle"
            },
            "'\n"
          ]
        ]
      }
    }
  }
}

By specifying order of execution of the configSets I am able to run everything I need to install and configure puppet and after that run the commands to install the plugin.

score 0 · Answer 2 · answered Nov 08 '22 at 09:33

0

You can consider using DependsOn attribute to enforce the order :- https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-attribute-dependson.html

answered Nov 08 '22 at 09:33

zipate

166
1
4

Cloudformation template order of execution?

2 Answers2