Data bag encryption encrypts on Chef server, but how to encrypt local copy?

Question

I have a full Chef configuration set of roles, cookbooks, databags, etc., in a private git repository.

I see that I can use the --secret-file option to encrypt a databag when it's uploaded to the Chef server.

But I want to store those databags encrypted in the git repository as well.

The only thing that comes to mind is making a plaintext json file locally, uploading it to Chef with encryption, then taking the encrypted JSON from the Chef web page and pasting it into my repository copy.

Has anyone else solved this problem?

Draco Ater · Accepted Answer · 2013-09-18T05:58:07.120

I have a following bash (called encrypted-databag.sh) in my chef working directory:

#!/bin/bash -e

knife data bag $1 $2 $3 --secret-file ~/.chef/encrypted_data_bag_secret
if [ "$1" == "edit" ] ; then
    knife data bag show $2 $3 -Fj > "./data_bags/$2/$3.json"
fi

It saves me typing every time I knife to show me the encrypted data bag. And it automatically updates/saves it into repository, when I edit it.

Updated on 30.08.2013

The drawback of the script above is that you edit your data bag straight on chef-server. But there is a problem when you are still working on some cookbook and haven't uploaded it, but the data bag already there and is used by the older version of the cookbook. This way when chef-client is run on some node, it may lead to some errors.

So I was thinking about editing the encrypted data bag locally, without chef-server and then upload the new version of it together with new version of cookbook (after the tests have passed). So here is the rake task I use now to edit encrypted data bags.

namespace 'databag' do
  desc 'Edit encrypted databag item.'
  task :edit, [:databag, :item, :secret_file] do |t, args|
    args.with_defaults :secret_file => "#{ENV['HOME']}/.chef/encrypted_data_bag_secret"
    secret = Chef::EncryptedDataBagItem.load_secret args.secret_file
    item_file = "data_bags/#{args.databag}/#{args.item}.json"
    tmp_item_file = "/tmp/#{args.databag}_#{args.item}.json"
    begin
      #decrypt data bag into tmp file
      raw_hash = Chef::JSONCompat.from_json IO.read item_file
      databag_item = Chef::EncryptedDataBagItem.new raw_hash, secret
      IO.write tmp_item_file, Chef::JSONCompat.to_json_pretty( databag_item.to_hash )
      #edit tmp file
      sh "#{ENV['EDITOR']} #{tmp_item_file}"
      #encrypt tmp file data bag into original file
      raw_hash = Chef::JSONCompat.from_json IO.read tmp_item_file
      databag_item = Chef::EncryptedDataBagItem.encrypt_data_bag_item raw_hash, secret
      IO.write item_file, Chef::JSONCompat.to_json_pretty( databag_item )
    ensure
      ::File.delete tmp_item_file #ensure tmp file deleted.
    end
  end
end

Now to edit encrypted data bag I use:

rake databag:edit[my_databag,item_in_databag]

Thanks, This helped me with a different but related problem, I can now encrypt databags directly from ruby without needing to contact the chef server to do it, so thanks, super helpful! if you wanted to know how I did it, then go here:http://stackoverflow.com/questions/26978737/encrypt-data-bag-with-knife-from-inside-ruby — Thermatix, Nov 17 '14 at 20:16

score 1 · Answer 2 · answered Nov 22 '13 at 06:08

I took the example from @draco-ater a little further as I would find myself wanting to edit data bags while in the cookbook directory by doing something like ../encrypted_data_bag etc so I shortened the name and used some extra bash magic to find the actual directory even if it's symlinked (can be called from anywhere in the filesystem). Since I'm not the only one working on the data bags I wanted the ability to show the data bag contents at any time in case someone else had updated items and still push the data into the repo (in case they forgot to add it). I also wanted the data to be deleted from the repository if it was deleted from the server to be more consistent. So I came up with this script. Please keep in mind that my encrypted_data_bag_secret resides under /etc/chef/encrypted_data_bag_secret, however others may choose to put it somewhere else which is why there is a SECRETFILE at the top of the script, just modify it with where your encrypted secret file exists.

#!/bin/bash -e
SECRETFILE="/etc/chef/encrypted_data_bag_secret"
SOURCE="${BASH_SOURCE[0]}"
while [ -h "$SOURCE" ] ; do SOURCE="$(readlink "$SOURCE")"; done
appDir="$( cd -P "$( dirname "$SOURCE" )/" && pwd )"
cd $appDir
if [ "$1" != "edit" ] && [ "$1" != "create" ] && [ "$1" != "delete" ] && [ "$1" != "show" ]
then
        echo "Bad arguments!"
        echo "First argument is what you want to do on the databag (show|edit|create|delete)"
        echo "Second argument is the databag"
        echo "Third argument is the item inside the databag"
        echo ""
        echo "Example: ./dbag edit aws main"
        echo ""
        echo "You used ($1) as the action to perform on ($2)/($3)"
        exit 1
else
        if [ "$1" == "delete" ]; then
                knife data bag $1 $2 $3
                if [ -z "$3" ]; then
                        rm -rf data_bags/$2
                else
                        rm -f data_bags/$2/$3.json
                fi
        else
                mkdir -p $appDir/data_bags/$2
                knife data bag $1 $2 $3 --secret-file $SECRETFILE
                knife data bag show $2 $3 -Fj > "./data_bags/$2/$3.json"
        fi
fi

Francois · Answer 3 · 2014-01-24T17:35:53.677

0

One other way to achieve this is to use knife plugins:

knife backup https://github.com/mdxp/knife-backup

knife backup export data_bags

knife essentials https://github.com/jkeiser/knife-essentials

knife download data_bags

edited Jan 24 '14 at 17:35

answered Jan 24 '14 at 17:08

Francois

1,851
1
13
15

Data bag encryption encrypts on Chef server, but how to encrypt local copy?

3 Answers3

Linked