start pod with root privilege on OpenShift

Question

I have an image that requires root privilege to start.

Now I'm trying to deploy it on OpenShift.

this is the deployment yaml I used to deploy it

apiVersion: apps/v1
kind: Deployment
metadata:
  name: xyz
  annotations:
   k8s.v1.cni.cncf.io/networks: macvlan-conf
spec:
  selector:
    matchLabels:
      name: xyz
  template:
    metadata:
      labels:
        name: xyz
    spec:
      containers:
      - name: xyz
        image: 172.30.1.1:5000/myproject/xyz@sha256:bf3d219941ec0de7f52f6babbca23e03cc2611d327552b08f530ead9ec627ec2
        imagePullPolicy: Always
        securityContext:
          capabilities:
            add:
              - ALL
          privileged: false
          allowPrivilegeEscalation: false
          runAsUser: 0
        serviceAccount: runasanyuid
        serviceAccountName: runasanyuid
        hostNetwork: true
        resources:
          limits:
            memory: "12000Mi"
          requests:
            memory: "6000Mi"
        ports:
        - containerPort: 2102
        command:
        - /usr/sbin/sshd -D

please note that I already created a SCC called 'scc-admin' to run the pods in the project I'm working on with any UID, as I know that OpenShift doesn't allow pods to start with root privilege by default.

kind: SecurityContextConstraints
apiVersion: v1
metadata:
  name: scc-admin
allowPrivilegedContainer: true
runAsUser:
  type: RunAsAny
seLinuxContext:
  type: RunAsAny
fsGroup:
  type: RunAsAny
supplementalGroups:
  type: RunAsAny
users:
- developer
groups:
- developer

that's what I found on the internet as a solution for my issue, but I guess it didn't work as well :(

[root@centos72_base ~]# oc get scc
NAME               PRIV      CAPS      SELINUX     RUNASUSER          FSGROUP     SUPGROUP    PRIORITY   READONLYROOTFS   VOLUMES
anyuid             true      []        MustRunAs   RunAsAny           RunAsAny    RunAsAny    10         false            [configMap downwardAPI emptyDir hostPath persistentVolumeClaim projected secret]
hostaccess         false     []        MustRunAs   MustRunAsRange     MustRunAs   RunAsAny    <none>     false            [configMap downwardAPI emptyDir hostPath persistentVolumeClaim projected secret]
hostmount-anyuid   false     []        MustRunAs   RunAsAny           RunAsAny    RunAsAny    <none>     false            [configMap downwardAPI emptyDir hostPath nfs persistentVolumeClaim projected secret]
hostnetwork        false     []        MustRunAs   MustRunAsRange     MustRunAs   MustRunAs   <none>     false            [configMap downwardAPI emptyDir persistentVolumeClaim projected secret]
nonroot            false     []        MustRunAs   MustRunAsNonRoot   RunAsAny    RunAsAny    <none>     false            [configMap downwardAPI emptyDir persistentVolumeClaim projected secret]
privileged         true      [*]       RunAsAny    RunAsAny           RunAsAny    RunAsAny    <none>     false            [*]
restricted         false     []        MustRunAs   MustRunAsRange     MustRunAs   RunAsAny    <none>     false            [configMap downwardAPI emptyDir persistentVolumeClaim projected secret]
scc-admin          true      []        RunAsAny    RunAsAny           RunAsAny    RunAsAny    <none>     false            [awsElasticBlockStore azureDisk azureFile cephFS cinder configMap downwardAPI emptyDir fc flexVolume flocker gcePersistentDisk gitRepo glusterfs iscsi nfs persistentVolumeClaim photonPersistentDisk portworxVolume projected quobyte rbd scaleIO secret storageOS vsphere]
[root@centos72_base ~]#

please also note that this image works fine with docker using the below command

docker run -d --network host --privileged --cap-add=ALL --security-opt seccomp=unconfined --name xyz 172.30.1.1:5000/myproject/xyz /usr/sbin/sshd -D

[root@centos72_base ~]# docker ps | grep xyz
793e339ff732   172.30.1.1:5000/myproject/xyz              "/usr/sbin/sshd -D"      About a minute ago   Up About a minute             xyz

and on OpenShift i get these errors with the deployment file I provided above

Error creating: pods "xyz-7966f58588-" is forbidden: unable to validate against any security context constraint: [spec.containers[0].securityContext.securityContext.runAsUser: Invalid value: 0: must be in the ranges: [1000140000, 1000149999] capabilities.add: Invalid value: "ALL": capability may not be added]

which means that i have to remove

          capabilities:
            add:
              - ALL

and

          runAsUser: 0

to start the pod

and when I remove them from the yaml file, I get a crash loopback error from the pod

so can anyone please help me with that

the default service account used to run pods in openshift is not privileged enough to run pods with uids below the specified range. you will need to create a service account with anyuid privileges and use the sa to run your pods. see the following link for further information: https://www.openshift.com/blog/managing-sccs-in-openshift — meaningqo, Jul 27 '21 at 10:44
If you're certain the default ServiceAccount from your deployment namespace can use your custom SCC, then the issue would be in your SCC, not allowing for privileged containers execution. Rather than creating your own SCC, allowing volumes usage without PVC, stuff we don't find in others SCC: maybe just add your SA to the `anyuid` SCC, which is meant to be used by privileged containers. Or the SA name you've added to your SCC is malformatted (should be `system:serviceaccount::`) — SYN, Jul 27 '21 at 10:45
Now, depending on your application, you may not need to run as root -- and on Kubernetes, it's definitely recommended not to. Though not knowing what we're talking about, it's hard to tell. Check application logs. Try outside of OpenShift first if that can help you figure out what's required starting without root. — SYN, Jul 27 '21 at 10:49
your SCC `users` only mentions `developper`? Try adding `system:serviceaccount::` to that list. Otherwise it won't be used, your deployment would still comply with the `restricted` SCC. (feel free to edit your original post, adding the full definition / in doubt, for clarity, ...). Unless your SA is member of the `developer` group as well? — SYN, Jul 27 '21 at 10:51
thanks a lot for your replies actually as mentioned, I already created a new SCC with the following yaml file `[root@centos72_base ~]# cat scc_admin.yaml` `kind: SecurityContextConstraints` `apiVersion: v1` `metadata:` `name: scc-admin` `allowPrivilegedContainer: true` `runAsUser:` `type: RunAsAny` `seLinuxContext:` ` type: RunAsAny` `fsGroup:` ` type: RunAsAny` `supplementalGroups:` ` type: RunAsAny` `users:` `- developer` `groups:` `- developer` — kareem.baher, Jul 27 '21 at 10:53
I've edited my question by adding the SCC yaml file I made in case you didn't understand my above comment :D — kareem.baher, Jul 27 '21 at 11:00

score 2 · Accepted Answer · answered Jul 27 '21 at 19:07

The SCC you created currently applies to the User developer, and the group developer.

Your deployment uses a ServiceAccount runasanyuid.

You need to edit your SecurityContextConstraint, allowing that ServiceAccount:

[...]
users:
- developer
- system:serviceaccount:<namespace-for-your-deployment>:runasanyuid
groups:
- developer

As a side note, instead of creating your own SCC, if you just need to run a privileged container, OpenShift ships with the anyuid one that you could have re-used - just adding your SA into its users list.

While the best practice would be to avoid privileged containers. Unless there is a good reason, you should not run processes as root - which has been true, long before OCI.

If your application needs to write stuff some place, you could use some emptyDir volumes. If your application tries to bind on privileged ports, you should be able to reconfigure it. If your application complains about not being able to resolve an username for its UID, you could look into nsswrapper. Granting privileges with SecurityContextConstraints or PodSecurityPolicies isn't the norm, they're meant to be exceptions, carefully thought.

If your Pod crashes when running as non-root, try to check its logs. If that doesn't help, try to oc debug -n <namespace> pod/<podname>, which should start a new Pod, opening a shell into it, where you would be able to execute its entrypoint yourself, maybe try another set of options, slightly change your configuration, ... until you would get it right. You could even try from your workstation, docker run --user=12435679 xxx: pass it a random UID and see how your container/application deals with it.

Thank you soooo much @SYN your answer helped a lot, I appreciate it — kareem.baher, Jul 28 '21 at 09:55
Creating a Pod directly, you may have to set that SCC annotation yourself. Creating Deployments / DeploymentConfigs / StatefulSets / DaemonSets: the controller in charge of those, is meant to add it for you. If it's not your case, then your SCC probably doesn't allow your SA to use it. The fact that allowing `system:authenticated` could "work" would suggest previous delegations did not allow the right SA. --- oh... and the time it took for me to answer: question's gone ... was wondering about openshift.io/scc annotations ... — SYN, Jul 28 '21 at 21:58

score 1 · Answer 2 · edited Jul 28 '21 at 13:33

This how I troubleshoot my issue and solved it:

I started the pod in the debug mode using

[root@centos72_base ~]# oc debug -n myproject pod/xyz-5b4875f8d7-n7m2g
Defaulting container name to xyz.
Use 'oc describe pod/xyz-5b4875f8d7-n7m2g-debug -n myproject' to see all of the containers in this pod.

Debugging with pod/xyz-5b4875f8d7-n7m2g-debug, original command: /usr/sbin/sshd -D
Waiting for pod to start ...
If you don't see a command prompt, try pressing enter.
sh-4.2# whoami
whoami: cannot find name for user ID 1000140000
sh-4.2$

I figured that the pod is not running with root privilege.

I had to check the SCC that my pod is working with so I executed

[root@centos72_base ~]# oc get pod xyz-5b4875f8d7-n7m2g -o yaml | grep openshift.io/scc
    openshift.io/scc: restricted

I saw that my pod was still in restricted SCC, so I had to change the default SCC for the cluster using

[root@centos72_base ~]# oc adm policy add-scc-to-group privileged system:authenticated
scc "privileged" added to groups: ["system:authenticated"]
[root@centos72_base ~]#

and now I can see that its changed to privileged (I know its not the best practice, but I just wanted to make sure that the pod will run, the best practice is to run the pod with non-root privilege)

and after running the deployment again, I check the SCC again and I found that its now running with privileged SCC

[root@centos72_base ~]# oc get pod xyz-5b4875f8d7-n7m2g -o yaml | grep openshift.io/scc
    openshift.io/scc: privileged

then I entered the pod in debug mode to check that I am running the pod with the root user

[root@centos72_base ~]# oc debug -n myproject pod/xyz-5b4875f8d7-n7m2g
Defaulting container name to xyz.
Use 'oc describe pod/xyz-5b4875f8d7-n7m2g-debug -n myproject' to see all of the containers in this pod.

Debugging with pod/xyz-5b4875f8d7-n7m2g-debug, original command: /usr/sbin/sshd -D
Waiting for pod to start ...
If you don't see a command prompt, try pressing enter.
sh-4.2# whoami
root
sh-4.2#

I don't recommend using this as default solution for this issue, I just using this for testing and making sure that my app is working on OpenShift

here are the list of commands that helped me: run pod in debug mode:

oc debug -n <project-name> pod/<pod-name>

run pod in debug mode with root privelage:

oc debug deployment/<deployment-name> --as-root -n <project-name>

list your Security context:

oc get scc

open pod's yaml file:

oc get pod <pod-name> -o yaml

to list its scc:

oc get pod <pod-name> -o yaml | grep openshift.io/scc

to set the default scc:

oc adm policy add-scc-to-group <scc-name> system:authenticated

?!? `oc adm policy add-scc-to-group privileged system:authenticated`. Right here. You've just allowed any container, any deployment in your cluster, to use the privileged SCC - disabling privileges escalation checks in general. Restarting existing containers, it's not unlikely they would switch user from their original UID to root. As pointed out in my answer, you should delegate those privileges to `system:serviceaccount::`. If it were a RedHat supported cluster, your support engineer would have had a stroke. — SYN, Jul 28 '21 at 21:49

start pod with root privilege on OpenShift

2 Answers2