6

I am currently writing a Helm chart for a multi-container application. We have a bunch of microservice containers (we call them "applications") that are very similar in the way they can be handled through K8s, and can (and thus should) be handled by the same Helm template to avoid duplicating things. On the other hand, it makes sense to be able to configure certain settings individually for the different applications (e.g., resource requests). I am currently doing something like this:

{{- $applications:= <obtain list of applications> }}
{{ range $app:= $applications }}
apiVersion: apps/v1
kind: StatefulSet
spec:
[...]
    spec:
      containers:
      - name: {{ $app }}
        image: {{ $.Values.image.registry }}mycompany/myproduct-{{ $app }}:{{ $.Values.image.version }}
[...]

What I would like to do now is be able to set some application specific values (take K8s resources.requests.memory or number of replicas as an example) that can be set individually inside values.yaml for each application, but falls back to a default if they are not set. The idea was to have such a section in values.yaml:

applications:
  default:
    replicas: 1
    resources:
      requests:
        memory: 512Mi
        cpu: 250m
  applicationA:
    resources:
      requests:
        memory: 7Gi
  applicationB:
    resources:
      requests:
        cpu: 500m
[...]

The idea is that I use the value from the "default" section for applications, unless there is an application-specific value specified in the section for the respective application. In my example, I have a custom value for the memory request of applicationA and one for the CPU request of application B, but all other values should come from the default section. There can also be applications where no application-specific settings are defined at all (in which case everything is coming from the default section of values.yaml). The "index" function seemed to allow me to do what I want, so in my template, I tried this (line break for readability):

resources:
  requests:
    memory: {{ default \
(index $.Values "applications" "default" "resources" "requests" "memory") \
(index $.Values "applications" $app "resources" "requests" "memory") }}

Since the index allows me to use the value of my loop variable "$app" to "dereference" a value from values.yaml, this actually works... alas, index fails with error calling index: index of nil pointer in those cases where I do not specify an appliation-specific value for one of the settings, and thus the dereferencing fails. What I would need is index to NOT fail in that situation, but just return an empty value, so that the default would kick in. Unfortunately, I cannot seem to find a way to do this.

But maybe there is a complete different way of handling this problem, so any ideas and suggestions are appreciated. Of course as a fallback I could just explicitly set every value for each application in values.yaml, but that just does not feel right.

PalatinateJ
  • 183
  • 1
  • 1
  • 10
  • Couldn't you have a generic chart representing an `application` with the defaults and use that as a parent chart? So then define charts specific to each of the apps you want which use the parent/generic chart as a subchart. Then each of these charts could override values that come from the parent/generic chart. Then you could create a further umbella chart that includes all of the specific charts as subcharts so that they can be installed in a single chart. See also https://stackoverflow.com/questions/52724344/is-there-a-concept-of-inheritance-for-kubernetes-deployments/52724658#52724658 – Ryan Dawson Oct 10 '18 at 14:53
  • Hi Ryan, thank you for you reply. That might be an alternative, but the application specific settings are meant to be user-adjustable (and thus - at least to my still very limited understandig of helm - means they should go to values.yaml). Most of the actual defaults of the different applications (except from things specific to K8s) are already set in the container itself and can be overridden via env vars. On the other hand, for things that differ, but are NOT meant to be user-adjustable (liveness and readyness probes come to mind - I haven't got these yet), the subchart approach fits. – PalatinateJ Oct 10 '18 at 15:27
  • As a user I can set any variables in any of values.yml files of any of the subcharts of the chart that I'm installing -https://github.com/helm/helm/issues/2927#issuecomment-328902868 I just have to put the name of the subchart in front of the variable when I reference it like in https://github.com/helm/helm/blob/master/docs/chart_template_guide/subcharts_and_globals.md#overriding-values-from-a-parent-chart I can do this as a user on the command-line as well as within the umbrella chart – Ryan Dawson Oct 10 '18 at 15:47

1 Answers1

17

I do something similar with my charts, but based on environment instead of application.

In my values.yaml:

env: dev

replicaCount:
  _default: 1
  staging: 2
  prod: 4

In my template:

  replicas: {{ pluck .Values.env .Values.replicaCount | first | default .Values.replicaCount._default }}

The pluck command returns a list, and first selects the first item from that list. In the event that nothing is returned, it falls back to the _default value. So when I run helm template --set env=staging, I get the configuration I set for my staging environment.

Your values.yaml file would look like this:

replicas:
  _default: 1
resources:
  requests:
    memory:
      _default: 512Mi
      applicationA: 7Gi
    cpu:
      _default: 250m
      applicationB: 500m

And your template would have something like this:

memory: {{ pluck .Values.app .Values.resources.requests.memory | first | default .Values.resources.requests.memory._default }}

Then you'd just need to add --set app=applicationA to your Helm command.

I think this makes the values.yaml more-or-less readable, but the template gets a bit ugly.

You could potentially use a combination of pluck with your original values file for something like this:

{{ pluck .Values.app .Values.applications | default .Values.applications.default | toYaml | indent N }}

I haven't tried that, so YMMV. (You'll definitely have to change N to match the level of indentation where you insert the block.) Just an idea. Seems like high potential for brittle config.

aayore
  • 663
  • 5
  • 12