1

I'm trying to set up a workflow to develop Chef cookbooks locally. We're currently using Chef Server with the provisioned nodes using chef-client.

As part of the new workflow, we want to be able to start using Vagrant to test cookbooks locally to avoid incurring in the costs of testing on a remote machine in a cloud.

I'm able to launch and provision a local Vagrant machine, but the one thing I'm not really sure how to do is to have Chef load the local version of the cookbook, but still talk to the Chef server for everything else (environments, roles, data bags, etc.), so I don't have to upload the cookbook via knife every time I make a change I want to test. Is this possible?

In other words, can I make chef-client talk to the local chef-zero server only for the cookbooks but to the remote Chef server for everything else? Or maybe a different approach that would yield the same effect? I'm open to suggestions.

UPDATE

I think an example will help to express what I'm looking for. I'm realizing that this may not really be what I need, but I'm curious about how to achieve it anyway. In this scenario, a recipe reads from a databag stored in the remote Chef server

metadata.rb

name             'proxy-cookbook'
version          '0.0.0'

.kitchen.yml

---
driver:
  name: vagrant

provisioner:
  name: chef_zero

platforms:
  - name: ubuntu-12.04

suites:
  - name: default
    run_list:
      - recipe[proxy-cookbook::default]
    attributes:

recipes/default.rb

...
key = data_bag_item("key", "main")
....

Now, I know I can create something along the lines of:

data_bags/main.json

{
  "id": "main",
  "key": "s3cr3tk3y"
}

And have my kitchen tests read from that data bag; but that is exactly what I'm trying to avoid. Is it possible to either:

  • Instruct test-kitchen to get the actual data bag from chef server,
  • Have chef-zero retrieve a temporary copy of the data bags for local tests, or
  • Quickly "dump" the contents of a remote Chef server locally?

I hope that makes sense. I can add some context if necessary.

user2066657
  • 444
  • 1
  • 4
  • 23
Pato Arvizu
  • 21
  • 1
  • 6
  • 1
    I suggest using [test-kitchen](http://kitchen.ci/). – StephenKing Jun 08 '15 at 19:28
  • @StephenKing I'm in the process of trying to implementing the use of test-kitchen as well, and it's really useful, but it looks like that only shifts the problem, since I'm now not able to talk to my chef-server for non-cookbook stuff, like databags and environments. I would like to be able to have test-kitchen talk to the Chef server for everything _except_ cookbooks, or maybe a different approach like having chef-zero proxy everything but the cookbooks, does that make sense? – Pato Arvizu Jun 08 '15 at 21:35

2 Answers2

1

Test kitchen is the best way to drive vagrant. It provides the integration you're looking for with chef zero. Enables you to completely emulate your production chef setup locally and test your cookbook against multiple platforms.

Test kitchen has replaced the older workflows I used to have chef development. Very well worthwhile learning.

Example

Generate a demo cookbook that installs java using the community cookbook. Tools like Berkshelf (to manage cookbook dependencies) and chef zero are setup automatically.

chef generate cookbook demo

Creates the following files:

└── demo
    ├── .kitchen.yml
    ├── Berksfile
    ├── metadata.rb
    ├── recipes
    │   └── default.rb
    └── test
        └── integration
            ├── default
            │   └── serverspec
            │       └── default_spec.rb

.kitchen.yml

Update the platform versions. Kitchen is told to use vagrant and chef zero. 

---
driver:
  name: vagrant

provisioner:
  name: chef_zero

platforms:
  - name: ubuntu-14.04
  - name: centos-6.6

suites:
  - name: default
    run_list:
      - recipe[demo::default]
    attributes:

Berksfile

This file controls how cookbook dependencies are managed. The special "metadata" setting tells Berkshelf to refer to the cookbook metadata file.

source 'https://supermarket.chef.io'

metadata

metadata.rb

Add the "apt" and "java" cookbooks as a dependencies:

name 'demo'
..
..

depends "apt"
depends "java"

recipes/default.rb

include_recipe "apt"
include_recipe "java"

test/integration/default/serverspec/default_spec.rb

Test for the installation of the JDK package

require 'spec_helper'

describe package("openjdk-6-jdk") do
  it { should be_installed }
end

Running the example

$ kitchen verify default-ubuntu-1404
-----> Starting Kitchen (v1.4.0)

       ..
       ..

       Package "openjdk-6-jdk"
         should be installed

       Finished in 0.1007 seconds (files took 0.268 seconds to load)
       1 example, 0 failures

       Finished verifying <default-ubuntu-1404> (0m13.73s).
-----> Kitchen is finished. (0m14.20s)

Update

The following example demonstrates using test kitchen with roles (works for data bags and other items you want loaded into chef-zero):

Community
  • 1
  • 1
Mark O'Connor
  • 76,015
  • 10
  • 139
  • 185
  • Thanks for the walk-through Mark, this is along the lines of what I've been playing with in the last couple of hours and it's certainly helpful, but it's not quite what I'm looking for. Maybe a concrete example will help: I have my cookbook "cookbookA", the run list is defined in .kitchen.yml is "recipe[CookbookA::default]". I make local changes and do kitchen converge to test them, that works. Now, the cookbook depends on an environment attribute. I know I can create an environment file and have kitchen read from that, but can I avoid that and read the actual environment attributes? – Pato Arvizu Jun 09 '15 at 16:11
  • The space in my comment above is limited, let me edit my question. – Pato Arvizu Jun 09 '15 at 16:16
  • Edited question with example, I hope it's clearer now. – Pato Arvizu Jun 09 '15 at 16:47
  • @PatoArvizu chef_zero uses the "test/integration" directory as it's chef repository. Create you environments and data bags there. For example using roles see the update above or this link: http://stackoverflow.com/questions/29812546/testing-chef-roles-and-environments/29860876#29860876 – Mark O'Connor Jun 10 '15 at 15:03
  • @marc-oconnor That's exactly what I was trying to avoid, the need to create the files for the environments, data bags, etc. I posted an answer below with a solution that worked for me. It might not be the optimal way to use test-kitchen, but that's what achieved what I was looking for. – Pato Arvizu Jun 10 '15 at 16:47
0

I think I found what I was looking for.

You can use knife to download the Chef server objects that you need. You can bootstrap this in .kitchen.yml so you don't have to do it manually every time.

.kitchen.yml

...
driver:
  name: vagrant
  pre_create_command: 'mkdir -p chef-server; knife download /data_bags /roles /environments --chef-repo-path chef-server/'
...
provisioner:
  name: chef_zero
  data_bags_path: chef-server/data_bags
  roles_path: chef-server/roles
  environments_path: chef-server/environments
  client_rb:
    environment: development
...

And then I just added the chef-server directory to .gitignore

.gitignore

chef-server/

There might be a less redundant way of doing this, but this works for me right now, and since I just wanted to document this, I'm leaving it like that.

Pato Arvizu
  • 21
  • 1
  • 6