55

I want to store files in Kubernetes Secrets but I haven't found how to do it using a yaml file.

I've been able to make it using the cli with kubectl:

kubectl create secret generic some-secret --from-file=secret1.txt=secrets/secret1.txt

But when I try something similar in a yaml:

apiVersion: v1
kind: Secret
metadata:
  name: some-secret
type: Opaque
data:
  secret1.txt: secrets/secret1.txt

I´ve got this error:

[pos 73]: json: error decoding base64 binary 'assets/elasticsearch.yml': illegal base64 data at input byte 20

I'm following this guide http://kubernetes.io/docs/user-guide/secrets/. It explains how to create a secret using a yaml but not how to create a secret from a file using yaml.

Is it possible? If so, how can I do it?

Tinkaal Gogoi
  • 4,344
  • 4
  • 27
  • 36
dgil
  • 2,318
  • 2
  • 23
  • 39
  • Did you get any way to do this using `yml` file as you mention in your question ? like without running `kubectl create secret generic some-secret --from-file=`? – Nilesh Apr 12 '20 at 19:35
  • 2
    Use `stringData:` for raw string input, and use `data:` for `base64` encoded data. – Daniel Stevens Oct 26 '21 at 03:42

6 Answers6

41

As answered on previous post, we need to provide the certificate/key encoded as based64 to the file.

Here is generic example for a certiticate (in this case SSL):

The secret.yml.tmpl:

    apiVersion: v1    

    kind: Secret
    metadata:
         name: test-secret
         namespace: default
    type: Opaque
    data:
        server.crt: SERVER_CRT
        server.key: SERVER_KEY

Pre-process the file to include the certificate/key:

sed "s/SERVER_CRT/`cat server.crt|base64 -w0`/g" secret.yml.tmpl | \
sed "s/SERVER_KEY/`cat server.key|base64 -w0`/g" | \
kubectl apply -f -

Note that the certificate/key are encoded using base64 without whitespaces (-w0).

For the TLS can be simply:

kubectl create secret tls test-secret-tls --cert=server.crt --key=server.key
aitorhh
  • 2,331
  • 1
  • 23
  • 35
  • 7
    It's worth noting (in case anyone misunderstands) that the files server.crt and server.key are typically files that contain a short header, a long string that is a base64 encoding broken up with some spaces and new lines and then a short footer. The string that gets inserted into the yaml is a further base64 encoding generated by running an encoder on those existing files. There is therefore ultimately 2 levels of base64 encoding. Attempting instead to just copy the strings from inside the files and using those directly (since they are already base64 encodings) does not work. – morechilli Jan 24 '19 at 14:48
31

You can use --dry-run flag to prepare YAML that contains data from your files.

kubectl create secret generic jwt-certificates --from-file=jwt-public.cer --from-file=jwt-private.pfx --dry-run=true  --output=yaml > jwt-secrets.yaml

Edit

Thanks to @Leopd for comment about API deprecation, new kubectl uses this command:

kubectl create secret generic jwt-certificates --from-file=jwt-public.cer --from-file=jwt-private.pfx --dry-run=client --output=yaml > jwt-secrets.yaml

On my machine I still have old kubectl version

szogun1987
  • 627
  • 6
  • 14
20

When using the CLI format you're using a generator of the YAML before posting it to the server-side.

Since Kubernetes is a client-server app with a REST API in between, and all actions need to be atomic, the posted YAML needs to contain the content of the linked file(s), and the best way to do that is by embedding them in base64 encoded format (in-line). It would be nice if the file could be otherwise embedded (indentation maybe could be used to create the boundaries of the file), but I haven't seen any working examples of that.

That said, putting a file reference on the YAML is not possible, there is no pre-flight rendering of the YAML to include the content.

mirekphd
  • 4,799
  • 3
  • 38
  • 59
Dimitris
  • 1,276
  • 10
  • 9
9

So I just learned a super useful k8s fundamental I missed, and then discovered it has a security vulnerability associated with it, and came up with a resolution.

TLDR:
You can have cleartext multiline strings/textfiles as secret.yaml's in your secret repo !!! :)

2023 Update: I removed recommendation of storing secrets in HashiVault, I no longer recommend it since it has too much overhead. Now my go to recommendations are:

  1. Storing Cleartext secrets in a manage service equivalent to the secrets management part of HashiVault. Like: AWS Secrest Manager / GCP Secrets Manager / Azure Key Vault
  2. Using Mozilla Sops and AWS/GCP/Azure KMS to store Sops/KMS Encrypted Secrest in Git Repos
    (I personally prefer SOPS as I find it a bit easier to store versioned config and secrets next to each other.)

cleartext-appsettings.secret.yaml
appsettings.Dummy.json is the default file name (key of the secret)
(I use the word default file name as you could override the file name by specifying some rarely used settings in a pod's yaml that mounts the secret as a volume.)
(and the clear text json code is the file contents (value of the secret)

apiVersion: v1
kind: Secret
metadata:
  name: appsettings
  namespace: api
type: Opaque
stringData:
  appsettings.Dummy.json: |-
    {
      "Dummy": {
        "Placeholder": {
          "Password": "blank"
        }
      }
    }

When I
kubectl apply -f cleartext-appsettings.secret.yaml
kubectl get secret appsettings -n=api -o yaml
The secret shows up cleartext in the annotation...

apiVersion: v1
data:
  appsettings.Dummy.json: ewogICJEdW1teSI6IHsKICAgICJQbGFjZWhvbGRlciI6IHsKICAgICAgIlBhc3N3b3JkIjogImJsYW5rIgogICAgfQogIH0KfQ==
kind: Secret
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"Secret","metadata":{"annotations":{},"name":"appsettings","namespace":"api"},"stringData":{"appsettings.Dummy.json":"{\n  \"Dummy\": {\n    \"Placeholder\": {\n      \"Password\": \"blank\"\n    }\n  }\n}"},"type":"Opaque"}
  creationTimestamp: 2019-01-31T02:50:16Z
  name: appsettings
  namespace: api
  resourceVersion: "4909"
  selfLink: /api/v1/namespaces/api/secrets/appsettings
  uid: f0629027-2502-11e9-9375-6eb4e0983acc


Apparently the yaml used to create the secret showing up in the annotation is expected behavior for kubectl apply -f secret.yaml since 2016/has been posted as a bug report, but issue closed without resolution/they're ignoring it vs fixing it.

If you're original secret.yaml is base64'd the annotation will at least be base64'd but in this scenario it's straight up non-base64'd human readable clear text.

The following counteracts the leak by imperatively deleting the annotation:

  • kubectl apply -f cleartext-appsettings-secret.yaml -n=api
  • kubectl annotate secret appsettings -n=api kubectl.kubernetes.io/last-applied-configuration-
neoakris
  • 4,217
  • 1
  • 30
  • 32
4

You can use secode to replace secret values with base64 encoded strings, by simply doing:

secode secrets.yaml > secrets_base64.yaml

It encodes all data fields and works with multiple secrets (kind:Secret) per yaml file, when defined in a list (kind: List).

Disclaimer: I'm the author

mayr
  • 451
  • 6
  • 14
  • 1
    A similar mechanism is to use the `stringData` key in your yaml file to provide non-base64 encoded data and allow Kubernetes to do the encoding automatically. https://kubernetes.io/docs/concepts/configuration/secret/ – Taylor D. Edmiston Jun 25 '19 at 22:37
0

For the Windows users in the room, use this for each of the .cer and .key (example shows the .key being encoded for insertion in to the YAML file):

$Content = Get-Content -Raw -Path C:\ssl-cert-decrypted.key

[Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($Content)) | Out-File -FilePath C:\ssl-cert-decrypted.key.b64

Open the new .b64 file and paste the (single line) output in to your YAML file - be aware that if checking in the YAML file to a source code repo with this information in it, the key would effectively be compromised since base64 isn't encryption.

Lewis
  • 244
  • 2
  • 6