9

I have build a docker image containing tshark (its an image I am going to use for doing various manual debugging from a kubernetes pod).

I have deployed a container in kubernetes running that image. But when I access the container and try to run tshark I get:

$ kubectl exec myapp-cbd49f587-w2swx -it bash
root@myapp-cbd49f587-w2swx:/# tshark -ni any -f "test.host" -w sample.pcap -F libpcap
Running as user "root" and group "root". This could be dangerous.
Capturing on 'any'
tshark: cap_set_proc() fail return: Operation not permitted

Googling that error:

https://www.weave.works/blog/container-capabilities-kubernetes/ https://unofficial-kubernetes.readthedocs.io/en/latest/concepts/policy/container-capabilities/

it seems I need to configure a securityContext for my container/pod. In my deployment.yaml I have added:

  containers:
     ...
  securityContext:
    capabilities:
      add:
        - NET_ADMIN

But when I apply that deployment I get:

error: error validating "deployment.yaml": error validating data: ValidationError(Deployment.spec.template.spec.securityContext): unknown field "capabilities" in io.k8s.api.core.v1.PodSecurityContext; if you choose to ignore these errors, turn validation off with --validate=false

Adding --validate=false removes the error but also means the securityContext is ignored.

What is preventing me from setting:

  securityContext:
    capabilities:
      add:
        - NET_ADMIN

Based on the guides I have found this should be fine.

I have also looked at (looks to be non free):

https://sysdig.com/blog/tracing-in-kubernetes-kubectl-capture-plugin/

so probably the right way is to use some tool like that (ksniff) or setup a sidecar container. But I am still curious to why I get the above error.

u123
  • 15,603
  • 58
  • 186
  • 303

2 Answers2

13

Looking specifically to the error, you posted only part of your manifest and looking to this we can see that you put securityContext: in the same level as containers::

  containers:
     ...
  securityContext:
    capabilities:
      add:
        - NET_ADMIN

It should be under containers: as as written in the documentation:

To add or remove Linux capabilities for a Container, include the capabilities field in the securityContext section of the Container manifest.

Example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: security-context-demo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: security-context-demo
  template:
    metadata:
      labels:
        app: security-context-demo
    spec:
      containers:
      - name: sec-ctx-4
        image: gcr.io/google-samples/node-hello:1.0
        securityContext:
          capabilities:
            add:
            - NET_ADMIN
Mark Watney
  • 5,268
  • 2
  • 11
  • 33
  • 1
    But isn't securityContext configurable at the pod level as well? per https://v1-17.docs.kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod @u123 – Treefish Zhang Aug 28 '21 at 20:33
  • 3
    Check [Yasir](https://stackoverflow.com/users/4703720/yasir-mir) answer, it's only possible at container level. – Mark Watney Aug 29 '21 at 12:00
  • Great catch! Confusingly though, only `capabilities` is so strict that it will cause an error if your put it on the level of `containers`, while other settings like `runAsUser` will actually work even then... Note also the reverse, i.e. `capabilities.drop` – mirekphd May 28 '23 at 17:20
5

Linux capabilities can be added only at container level security-context, not at the pod level.

Not obvious here but see that the section on adding capabilities only mentions adding it to the container:

https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-capabilities-for-a-container

Yasir Mir
  • 63
  • 1
  • 4