How do you mock OpsWorks specific services/dependencies when developing locally with Kitchen and Chef?

Question

I'm writing Chef wrappers around some of the built in OpsWorks cookbooks. I'm using Berkshelf to clone the OpsWorks cookbooks from their github repo.

This is my Berksfile:

source 'https://supermarket.getchef.com'

metadata

def opsworks_cookbook(name)
  cookbook name, github: 'aws/opsworks-cookbooks', branch: 'release-chef-11.10', rel: name
end

%w(dependencies scm_helper mod_php5_apache2 ssh_users opsworks_agent_monit
   opsworks_java gem_support opsworks_commons opsworks_initial_setup
   opsworks_nodejs opsworks_aws_flow_ruby
   deploy mysql memcached).each do |cb|
  opsworks_cookbook cb
end

My metadata.rb:

depends 'deploy'
depends 'mysql'
depends 'memcached'

The problem is, when I try to override attributes that depend on the opsworks key in the node hash, I get a:

NoMethodError
-------------
undefined method `[]=' for nil:NilClass

OpsWorks has a whole bunch of pre-recipe dependencies that create these keys and do a lot of their setup. I'd like to find a way to either pull in those services and run them against my Kitchen instances or mock them in a way that I can actually test my recipes.

Is there a way to do this?

Peter M · Answer 1 · 2015-10-12T14:10:30.557

I would HIGHLY recommend you check out Mike Greiling's blog post Simplify OpsWorks Development With Packer and his github repo opsworks-vm which help you to mock the entire opsworks stack including the install of the opsworks agent so you can also test app deploy recipes, multiple layers, multiple instances at the same time, etc . It is extremely impressive.

Quick Start on Ubuntu 14.04

NOTE: This can NOT be done from an ubuntu virtual machine because virtualbox does not support nested virtualization of 64-bit machines.

Install ChefDK
1. mkdir /tmp/packages && cd /tmp/packages
2. wget https://opscode-omnibus-packages.s3.amazonaws.com/ubuntu/12.04/x86_64/chefdk_0.8.1-1_amd64.deb
3. sudo dpkg -i chefdk_0.8.0-1_amd64.deb
4. cd /opt/chefdk/
5. chef verify
6. which ruby
7. echo 'eval "$(chef shell-init bash)"' >> ~/.bash_profile && source ~/.bash_profile
Install VirtualBox
1. echo 'deb http://download.virtualbox.org/virtualbox/debian vivid contrib' > /etc/apt/sources.list.d/virtualbox.list
2. wget -q https://www.virtualbox.org/download/oracle_vbox.asc -O- | sudo apt-key add -
3. sudo apt-get update -qqy
4. sudo apt-get install virtualbox-5.0 dkms
Install Vagrant
1. cd /tmp/packages
2. wget https://dl.bintray.com/mitchellh/vagrant/vagrant_1.7.4_x86_64.deb
3. sudo dpkg -i vagrant_1.7.4_x86_64.deb
4. vagrant plugin install vagrant-berkshelf
5. vagrant plugin install vagrant-omnibus
6. vagrant plugin list
Install Packer
1. mkdir /opt/packer && cd /opt/packer
2. wget https://dl.bintray.com/mitchellh/packer/packer_0.8.6_linux_amd64.zip
3. unzip packer_0.8.6_linux_amd64.zip
4. echo 'PATH=$PATH:/opt/packer' >> ~/.bash_profile && source ~/.bash_profile
Build Mike Greiling's opsworks-vm virtualbox image using Packer
1. mkdir ~/packer && cd ~/packer
2. git clone https://github.com/pixelcog/opsworks-vm.git
3. cd opsworks-vm
4. rake build install
5. This will install a new virtualbox vm to ~/.vagrant.d/boxes/ubuntu1404-opsworks/

To mock a single opsworks instance, create a new Vagrantfile like so:

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu1404-opsworks"
  config.vm.provision :opsworks, type: 'shell', args: 'path/to/dna.json'
end

The dna.json file path is set relative to the Vagrantfile and should contain any JSON data you wish to send to OpsWorks Chef.

For example:

{
  "deploy": {
    "my-app": {
      "application_type": "php",
      "scm": {
        "scm_type": "git",
        "repository": "path/to/my-app"
      }
    }
  },
  "opsworks_custom_cookbooks": {
    "enabled": true,
    "scm": {
      "repository": "path/to/my-cookbooks"
    },
    "recipes": [
      "recipe[opsworks_initial_setup]",
      "recipe[dependencies]",
      "recipe[mod_php5_apache2]",
      "recipe[deploy::default]",
      "recipe[deploy::php]",
      "recipe[my_custom_cookbook::configure]"
    ]
  }
}

To mock multiple opsworks instances and include layers see his AWS OpsWorks "Getting Started" Example which includes the stack.json below.

Vagrantfile (for multiple instances)

Vagrant.configure("2") do |config|

  config.vm.box = "ubuntu1404-opsworks"

  # Create the php-app layer
  config.vm.define "app" do |layer|

    layer.vm.provision "opsworks", type:"shell", args:[
      'ops/dna/stack.json',
      'ops/dna/php-app.json'
    ]

    # Forward port 80 so we can see our work
    layer.vm.network "forwarded_port", guest: 80, host: 8080
    layer.vm.network "private_network", ip: "10.10.10.10"
  end

  # Create the db-master layer
  config.vm.define "db" do |layer|

    layer.vm.provision "opsworks", type:"shell", args:[
      'ops/dna/stack.json',
      'ops/dna/db-master.json'
    ]

    layer.vm.network "private_network", ip: "10.10.10.20"
  end
end

stack.json

{
  "opsworks": {
    "layers": {
      "php-app": {
        "instances": {
          "php-app1": {"private-ip": "10.10.10.10"}
        }
      },
      "db-master": {
        "instances": {
          "db-master1": {"private-ip": "10.10.10.20"}
        }
      }
    }
  },
  "deploy": {
    "simple-php": {
      "application_type": "php",
      "document_root": "web",
      "scm": {
        "scm_type": "git",
        "repository": "dev/simple-php"
      },
      "memcached": {},
      "database": {
        "host": "10.10.10.20",
        "database": "simple-php",
        "username": "root",
        "password": "correcthorsebatterystaple",
        "reconnect": true
      }
    }
  },
  "mysql": {
    "server_root_password": "correcthorsebatterystaple",
    "tunable": {"innodb_buffer_pool_size": "256M"}
  },
  "opsworks_custom_cookbooks": {
    "enabled": true,
    "scm": {
      "repository": "ops/cookbooks"
    }
  }
}

For those not familiar with vagrant you just do a vagrant up to start the instance(s). Then you can modify your cookbook locally and any changes can be applied by re-running chef against the existing instance(s) with vagrant provision. You can do a vagrant destroy and vagrant up to start from scratch.

score 0 · Accepted Answer · answered Sep 19 '14 at 19:08

0

You'll have to do it manually. You can add arbitrary attributes to your .kitchen.yml, just go on an OpsWorks machine and log the values you need and either use them directly or adapt them to workable test data.

answered Sep 19 '14 at 19:08

coderanger

52,400
4
52
75

That still doesn't really solve the problem of testing against recipes that depend on Amazon's built in recipes. Are you suggesting that I use Berkshelf to pull in all of the OpsWorks recipes, run them, then run my recipes? What about if I want to, for example, install `mysql::client` which requires `node[:opsworks][:layers][:db-master]` or `node[:opsworks][:stack][:rds_instances][i][:engine] == 'mysql'`? I understand how to run these recipes, but how do I test against them using chefspec? – davepgreene Sep 20 '14 at 19:52
1

Yes, your existing Berksfile is correct as far as pulling in the right cookbooks. ChefSpec is just unit testing so dependencies shouldn't generally matter, but you would do the same thing as with Test Kitchen with the canned node data. – coderanger Sep 20 '14 at 20:07
1

I understand. I'm marking this as the answer because you've pointed me in the right direction. Thanks! – davepgreene Sep 22 '14 at 15:46

How do you mock OpsWorks specific services/dependencies when developing locally with Kitchen and Chef?

2 Answers2

Quick Start on Ubuntu 14.04