82

The settings

Consider an Ansible inventory file similar to the following example:

[san_diego]
host1
host2

[san_francisco]
host3
host4

[west_coast]
san_diego
san_francisco

[west_coast:vars]
db_server=foo.example.com
db_host=5432
db_password=top secret password

The problem

I would like to store some of the vars (like db_password) in an Ansible vault, but not the entire file.

How can a vault-encrypted ansible file be imported into an unencrypted inventory file?

What I've tried

I have created an encrypted vars file and tried importing it with:

include: secrets

To which ansible-playbook responded with:

ERROR: variables assigned to group must be in key=value form

Probably because it tried to parse the include statement as a variable.

Adam Matan
  • 128,757
  • 147
  • 397
  • 562

6 Answers6

120

Since Ansible 2.3 you can encrypt a Single Encrypted Variable. IMO, a walkthrough is needed as the doco's seem pretty terse.

Given an example of: mysql_password: password123 (within main.yml)

Run a command such as:

ansible-vault encrypt_string password123 --ask-vault-pass

This will produce:

    !vault |
$ANSIBLE_VAULT;1.1;AES256
66386439653236336462626566653063336164663966303231363934653561363964363833
3136626431626536303530376336343832656537303632313433360a626438346336353331
Encryption successful

paste this into your main.yml:

mysql_password: !vault |
    $ANSIBLE_VAULT;1.1;AES256
    66386439653236336462626566653063336164663966303231363934653561363964363833
    3136626431626536303530376336343832656537303632313433360a626438346336353331

run playbook:

Ie, ansible-playbook -i hosts main.yml --ask-vault-pass

Verify via debug:

- debug:
    msg: "mysql Pwd: {{ mysql_password }}"
peedee
  • 3,257
  • 3
  • 24
  • 42
wired00
  • 13,930
  • 7
  • 70
  • 73
  • 7
    As of Aug 2018 this should be considered the correct answer to the stated question above. – ncrmro Aug 01 '18 at 15:49
  • 3
    Is there a way that you can decrypt all of the vault values at once without having to copy/paste each encrypted string and run the reverse of encrypt_string? – Joe J Nov 01 '18 at 14:24
  • what type of formatting is that? is that a newline? does it matter? – jouell Mar 26 '19 at 18:56
  • @jouell yeah those are simply newlines with spaces for indents. But doesn't seem to matter. I guess whatever yaml supports – wired00 May 02 '19 at 02:04
  • Does this solution work well when the playbook is committed to git? – Martin Bjeldbak Madsen Sep 23 '19 at 11:02
  • @MartinBjeldbakMadsen how do you mean committed to git? Are you referring to issues with indentation of yml or something? I would hope that whoever is using any playbook including the above, would have it committed to git! But yes, no issues with git commit – wired00 Sep 24 '19 at 06:07
  • Doesn't this leave the password written in plain text in `history`? – Charlie Nov 11 '19 at 16:16
  • what if i have encrypted secret1 with different password from secret2, How to run playbook with two different password? – letthefireflieslive Nov 14 '19 at 02:31
  • @letthefireflieslive you should just have a single ansible vault password afaik – wired00 Nov 21 '19 at 06:51
  • @charlie yah I guess so. Just run `history -c` but really this should be running from an ephemeral pipeline agent such as good or Jenkins etc. But if being run from you local machine then yah `history -c` to clear history – wired00 Nov 21 '19 at 06:53
  • @wired00 okay makes sense, but, if ran from Jenkins how would you prompt for the password? Perhaps create a password file on Jenkins called `~/vault_password` and use the argument `vault_id @~/vault_password`? – Charlie Nov 22 '19 at 10:03
  • @charlie I'm more familiar with gocd but you can use credentials in Jenkins I believe https://ryandaniels.ca/blog/ansible-vault-jenkins/ – wired00 Nov 28 '19 at 17:58
62

If your issue is to have both unencrypted and encrypted vars files per group_hosts.

You can use this ansible feature : http://docs.ansible.com/ansible/playbooks_best_practices.html#best-practices-for-variables-and-vaults

group_vars/ 
  san_diego/
    vars.yml  # unencrypted yaml file
    vault.yml # encrypted yaml file

Ansible will read automatically vault.yml as encrypted yaml file.

Update : The solution below is also good solution (since Ansible 2.3)

Antoine
  • 4,456
  • 4
  • 44
  • 51
  • Meaning that san_diego is a directory, rather than a file? – Adam Matan Oct 27 '15 at 16:47
  • 2
    Exactly, you can use san_diego.yml or san_diego/vars.yml, that's the same. – Antoine Oct 27 '15 at 17:01
  • You are looking for this answer! @AdamMatan pls mark it as correct. – Raz Jan 29 '16 at 15:50
  • The sited best practices suggest naming the files `vars` and `vault`, sans the `.yml` suffix. In any case `vault.yml` seems odd b/c it's an encrypted file not a YAML file. – kkurian Jun 18 '16 at 01:02
  • The contents of the `vault.yml` file should be YAML, with the file itself then vaulted. No different than any other vault-encrypted file. – ntwrkguru Jun 10 '20 at 18:31
  • 1
    I'd argue this is a better solution than the one below as key rotation is easier with file level encryption. In all cases, know the pros and cons of file-level vs variable-level encryption and choose wisely. – disklosr Jul 13 '20 at 21:29
  • Will this work in a role, in the role's *default* or *vars* directories? For instance, I'd like `repo_root/roles/my_role/vars/main.yml` to stand in for `vars.yml` above, and `repo_root/roles/my_role/vars/vault.yml` to stand in for `vault.yml` above. The only difference is that they are not in a group_vars directory, and obviously main.yml is named differently. – Life5ign Dec 18 '20 at 21:10
19

At this time with Ansible 2.3 it's possible to have in a plain yaml both encrypted and unencrypted variables. The format of the variables encrypted is like this:

dbServer: PlainDatabaseServer
dbName: PlainDatabaseName
dbUser: PlainUser
dbPasswd: !vault |
      $ANSIBLE_VAULT;1.1;AES256
      63633363616165656538656537323835343634633063386137353637646663333939623464666437
      6263383933656635316436313934366564316337623435350a386362613838373363393534383232
      39663162363066313431623466363763356466376538613532333731613538373431623239626330
      6463373238366630360a623566616535376339326431363465663431623462356238636333306663
      6439

You can encrypt the variable using a password or a password file with the statement:

ansible-vault encrypt_string "dummy" --vault-password-file pass-ansible.txt

This statement returns the text shown in dbPasswd variable in the yaml above.

To run a playbook that uses the encrypted variable just add the following var:

 ansible-playbook playbooks/myplaybook --vault-password-file pass-ansible.txt

Or you can do the same with --ask-vault-pass which ask you for the password when executing the playbook:

ansible-playbook playbooks/myplaybook --ask-vault-pass
V. Morate
  • 207
  • 2
  • 5
  • 1
    I think you should also mention --ask-vault-pass, not only --vault-password-file – ympostor Jun 07 '17 at 07:13
  • Yes, you're right, thanks! We just use --vault-password-file because we run the playbooks with Jenkins in an automated way and it's simpler for us using a file with the password instead of passing the password via pipes or prompting. I add your option to the post. – V. Morate Jun 08 '17 at 08:06
  • Thanks for actually answering the question! – Tobias Oct 19 '18 at 23:39
  • Thanks for this solution! I didn't know this was an option. I dug in a little more and used this after reading it. Here's a slightly more verbose blog post I made on it afterward in case anyone wants a fuller example a little too verbose for SO -> https://coding-stream-of-consciousness.com/2019/05/08/ansible-vault-encrypt-single-strings-in-config-files-not-whole-file/. – John Humphreys May 10 '19 at 20:47
  • Just in case this is useful to anyone, to view an inline encrpted value in a yaml file, I found `yq` very handy: `yq -r '.users[]|select(.name == "fred").password' config.yml | ansible-vault decrypt --vault-password-file vault-password`. Of course that's an example that will need modifying according to one's own yaml structure. I found out about `yq` [here](https://dev.to/vikcodes/yq-a-command-line-tool-that-will-help-you-handle-your-yaml-resources-better-8j9), – starfry Feb 06 '23 at 10:09
7

You can do something similar to this.

  1. Create a password file (a plain text file with your password on a single line)

  2. Create an ansible.cfg in your ansible project folder

    [defaults]
vault_password_file = <path/to/your/password/file>

  3. Create a playbook file (e.g. playbook.yml)

     - name: my ansible playbook
   hosts: 127.0.0.1
   vars_files:
     - 'vars.yml'
   tasks:
     - name: print secure variable
       debug: msg="my secure variable '{{ my_secure_variable }}'"`

  4. Create a variable file (e.g. vars.yml)

    my_secure_variable: "X_my_secret_X"

  5. Encrypt the variable file (from the ansible project location with the ansible.cfg)

    ansible-vault encrypt vars.yml

  6. Run your playbook (from the ansible project location with the ansible.cfg)

    ansible-playbook -i "localhost," playbook.yml

You should get output similar to:

$ ansible-playbook playbook.yml -i 'localhost,'

PLAY [my ansible playbook] ****************************************************

GATHERING FACTS ***************************************************************

ok: [127.0.0.1]

TASK: [print secure variable] *************************************************

ok: [127.0.0.1] => {
    "msg": "my secure variable 'X_my_secret_X' "
}

PLAY RECAP ********************************************************************

127.0.0.1                  : ok=2    changed=0    unreachable=0    failed=0
grandma
  • 263
  • 3
  • 6
  • Where are the non-encrypted variables stored? – Adam Matan Jun 23 '15 at 19:39
  • Ansible is fairly flexible regarding variable placement and order of precedence. See http://docs.ansible.com/playbooks_variables.html#variable-precedence-where-should-i-put-a-variable. If we expand on the previous example, we can (but not limited to) do something like this: `- name: my ansible playbook hosts: 127.0.0.1 vars_files: - 'vars.yml' - 'insecure.yml' - ... ` Where insecure.yml is yet another yaml file with vars that simply have not been vaulted. You can also directly embed vars in the playbook with the `vars:` keyword. – grandma Jun 25 '15 at 03:50
  • 2
    In other words, this doesn't answer the question, which was "how to encrypt _some variables_". – Robin Daugherty Mar 06 '19 at 16:06
  • @RobinDaugherty do you have some insight that helps clarify your statement? Otherwise to respond to your statement, you place a key/value pair in a clear-text file you wish to "vault". When you run Ansible vault it converts that file into an encrypted key store. For more info, perhaps RTFM here: https://docs.ansible.com/ansible/latest/user_guide/vault.html#what-can-be-encrypted-with-vault. Once vaulted, you can access that variable like you would any other, however at rest it is encrypted. – grandma Mar 08 '19 at 05:15
4

It depends on your workflow. You can use a group_vars file as per Sebastian Stigler suggestion or if you want to use an inventory file, you can just add another "ini-like" file in an inventory directory and encrypt it.

$ mkdir my_inventory/
$ cat >> hosts << EOF
[san_diego]
host1
host2

[san_francisco]
host3
host4

[west_coast]
san_diego
san_francisco
EOF

$ cat >> inventory_crypted_vars << EOF
[west_coast:vars]
db_server=foo.example.com
db_host=5432
db_password=top secret password
EOF

Then, use -i my_inventory/ in your command line, or create a local ansible.cfg containing:

[defaults]
hostfile = ./my_inventory/

and you should be set. Ansible will merge both files at run time.

Use ansible-vault encrypt my_inventory/inventory_crypted_vars before committing and you're set.

You probably want a pre-commit hook to ensure that you're not committing unencrypted version of the file. For instance a pre-commit hook like this would do the trick (adjust FILES_PATTERN accordingly).

slm
  • 15,396
  • 12
  • 109
  • 124
leucos
  • 17,661
  • 1
  • 44
  • 34
0

You can use group_vars (see http://docs.ansible.com/playbooks_variables.html#variable-precedence-where-should-i-put-a-variable).

Create a subdirectory in your playbook named group_vars.
There you create a file named west_coast and put the following entries in it:

---
db_server: foo.example.com
db_host: 5432
db_password: top secret password

This file can then be converted to an ansible vault.

Sebastian Stigler
  • 6,819
  • 2
  • 29
  • 31
  • The problem with this approach is that the variables neither be accessible from other playbook, nor change when another inventory file is specified with the `-i` argument. – Adam Matan May 13 '15 at 16:17
  • You can also put this file to the global config directory `/etc/ansible/group_vars` instead of the directory `./group_vars` in the playbook (see the link in my answer). – Sebastian Stigler May 13 '15 at 18:48
  • Doesn't answer the question of some vars within a file being encrypted and some not. – ntwrkguru Jun 10 '20 at 18:52
  • If you read all the other answers you can see, that it is not possible to do it in an single file. – Sebastian Stigler Jun 11 '20 at 22:03