61

I am trying to configure Kubernetes RBAC in the least-permissive way possible and I want to scope my roles to specific resources and subresouces. I've dug through the docs and can't find a concise list of resources and their subresources.

I'm particularly interested in a the subresource that governs a part of a Deployment's spec--the container image.

Tanveer Alam
  • 5,185
  • 4
  • 22
  • 43
Chris Snell
  • 913
  • 2
  • 7
  • 11
  • Didi you manage to find an answer to this? – John Jul 11 '18 at 15:27
  • Disregard my comment, see my answer below for listing out all resources and verbs. – John Jul 12 '18 at 12:28
  • This script https://github.com/tanalam2411/kubernetes/blob/master/k8s_api/k8s_api_conventions/list_resource_n_subresources.go generates md(https://github.com/tanalam2411/kubernetes/blob/master/k8s_api/k8s_api_conventions/resources.md) file and also serves list of resources and sub resources over http. – Tanveer Alam Dec 23 '19 at 13:21

12 Answers12

93

Using kubectl api-resources -o wide shows all the resources, verbs and associated API-group.

$ kubectl api-resources -o wide
NAME                              SHORTNAMES     APIGROUP                       NAMESPACED   KIND                             VERBS
bindings                                                                        true         Binding                          [create]
componentstatuses                 cs                                            false        ComponentStatus                  [get list]
configmaps                        cm                                            true         ConfigMap                        [create delete deletecollection get list patch update watch]
endpoints                         ep                                            true         Endpoints                        [create delete deletecollection get list patch update watch]
events                            ev                                            true         Event                            [create delete deletecollection get list patch update watch]
limitranges                       limits                                        true         LimitRange                       [create delete deletecollection get list patch update watch]
namespaces                        ns                                            false        Namespace                        [create delete get list patch update watch]
nodes                             no                                            false        Node                             [create delete deletecollection get list patch update watch]
persistentvolumeclaims            pvc                                           true         PersistentVolumeClaim            [create delete deletecollection get list patch update watch]
persistentvolumes                 pv                                            false        PersistentVolume                 [create delete deletecollection get list patch update watch]
pods                              po                                            true         Pod                              [create delete deletecollection get list patch update watch]
statefulsets                      sts            apps                           true         StatefulSet                      [create delete deletecollection get list patch update watch]
meshpolicies                                     authentication.istio.io        false        MeshPolicy                       [delete deletecollection get list patch create update watch]
policies                                         authentication.istio.io        true         Policy                           [delete deletecollection get list patch create update watch]
...
...

I guess you can use this to create the list of resources needed in your RBAC config

Ryan
  • 4,594
  • 1
  • 32
  • 35
Doctor
  • 7,115
  • 4
  • 37
  • 55
  • This does not list sub-resources, like e.g. `service/proxy`, which can be used for RBAC. – dastrobu Dec 13 '21 at 10:39
  • 6
    There has been an update to the output of this command. Now the `APIGROUP` column has been renamed to `APIVERSION` and contains the concatenation of the ` + "/" + `. So if you are looking for the group (ie for RBAC) then you just need to take the part left of the `/`. – Doctor Jun 19 '22 at 11:49
30

The resources, sub-resources and verbs that you need to define RBAC roles are not documented anywhere in a static list. They are available in the discovery documentation, i.e. via the API, e.g. /api/apps/v1.

The following bash script will list all the resources, sub-resources and verbs in the following format:

api_version resource: [verb]

where api-version is core for the core resources and should be replaced by "" (an empty quoted string) in your role definition.

For example, core pods/status: get patch update.

The script requires jq.

#!/bin/bash
SERVER="localhost:8080"

APIS=$(curl -s $SERVER/apis | jq -r '[.groups | .[].name] | join(" ")')

# do core resources first, which are at a separate api location
api="core"
curl -s $SERVER/api/v1 | jq -r --arg api "$api" '.resources | .[] | "\($api) \(.name): \(.verbs | join(" "))"'

# now do non-core resources
for api in $APIS; do
    version=$(curl -s $SERVER/apis/$api | jq -r '.preferredVersion.version')
    curl -s $SERVER/apis/$api/$version | jq -r --arg api "$api" '.resources | .[]? | "\($api) \(.name): \(.verbs | join(" "))"'
done

WARNING: Note that where no verbs are listed via the api, the output will just show the api version and the resource, e.g.

core pods/exec:

In the specific instance of the following resources, no verbs are shown via the api, which is wrong (Kubernetes bug #65421, fixed by #65518):

nodes/proxy
pods/attach
pods/exec
pods/portforward
pods/proxy
services/proxy

The supported verbs for these resources are as follows:

nodes/proxy: create delete get patch update
pods/attach: create get
pods/exec: create get
pods/portforward: create get
pods/proxy: create delete get patch update
services/proxy: create delete get patch update

WARNING 2: Sometime Kubernetes checks for additional permissions using specialised verbs that are not listed here. For example, the bind verb is needed for roles and clusterroles resources in the rbac.authorization.k8s.io API group. Details of these specialised verbs are to be found in the docs here.

Valéry
  • 4,574
  • 1
  • 14
  • 25
John
  • 10,837
  • 17
  • 78
  • 141
9

I hesitate to even put this as an "Answer", but it is for sure too long for a comment

For the list of resources, are you aware of $HOME/.kube/cache/discovery wherein the Swagger JSON files are persisted to disk by directory that matches their enclosing apiVersion? This is the fastest link I could find (look in the "Discovering and Using CRDs" heading) but ls -la ~/.kube/cached/discovery will show what I mean. Those Swagger JSON files enumerate all the major players within an apiVersion in a way that I find a lot more accessible than the API reference website.

I don't have those files in front of me to know if they contain subresource definitions, so hopefully someone else can weigh in on that.

The minor asterisk to the "weigh in" part is that, based on the surfing I did of the RBAC docs and the 1.9 API reference, I didn't get the impression that a subresource is "field level access" to its parent resource. For example, v1beta1/Evictions is a Pod subresource of /evictions which to the best of my knowledge is not a field within PodSpec

So if you are interested in doing RBAC to constrain a Deployment's image, you may be much happier with Webhook Mode where one can have almost unbounded business logic applied to the attempted request.

mdaniel
  • 31,240
  • 5
  • 55
  • 58
7
for kind in `kubectl api-resources | tail +2 | awk '{ print $1 }' | sort`; do kubectl explain $kind ; done | grep -e "KIND:" -e  "VERSION:" | awk '{print $2}' | paste -sd' \n'
cs95
  • 379,657
  • 97
  • 704
  • 746
Ashish Kumar
  • 524
  • 6
  • 18
  • 1
    I had to modify a little to get it run on my mac: `for kind in $(kubectl api-resources | tail -n +2 | awk '{ print $1 }' | sort); do kubectl explain $kind ; done | egrep "(KIND|VERSION)" | awk '{print $2}' | paste -sd' \n'` – Robert Ranjan Jun 14 '20 at 23:57
  • 1
    Both commands does not work on mac: `usage: paste [-s] [-d delimiters] file ...` – Dentrax May 23 '22 at 12:17
  • Not that clean or pretty but works for specific use cases. – Akito Jul 06 '22 at 11:29
  • It was working on mac before, seems problem with ```paste``` command parameters on latest MAC OS version. Try the new one- ```for kind in $(kubectl api-resources | tail -n +2 | awk '{ print $1 }' | sort); do kubectl explain $kind ; done | egrep "(KIND|VERSION)" | awk '{print $2}' | paste -d " " - -``` Requesting all friends to modify accordingly if in future any of used command gets upgraded. – Ashish Kumar Jul 06 '22 at 18:35
4

You can find the resources list of Kubernetes v1.26 from here: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/. For other K8s versions, check https://kubernetes.io/docs/reference/kubernetes-api/

Check the catalog on the left side, for example, 'Workloads' is the high-level overview of the basic types of resources such as Container, Deployment, CronJob etc. And these subresources like 'Container, Deployment, CronJob' are the typical basic Kubernetes API resources.

You can access these basic resources via kubectl, hence there also have a list of 'Resource types' available in https://kubernetes.io/docs/reference/kubectl/cheatsheet/

But I'm confusing in your statement "a the subresource that governs a a part of a Deployment's spec--the container image", if you are trying to manage the permissions of an container image, you should do it on your image registry, but not on Kubernetes side. For example, your registry should has an access controller to do authentication when user pulling images.

Aleksey Kontsevich
  • 4,671
  • 4
  • 46
  • 101
Haoming Zhang
  • 2,672
  • 2
  • 28
  • 32
  • OK, getting close. It's still not obvious from the API docs what the sub-resources of the Deployment resource are. Specifically, I'm interested in referencing the image of a deployment's templated pods. I have to go deep down in a nest of resources to get there: Deployment -> DeploymentSpec -> PodTemplate -> PodSpec -> Container -> image. – Chris Snell Mar 21 '18 at 18:58
  • What I'm trying to do is create a role that can update the image tag of a Deployment's pods, but not the image itself. If a role can update the entire image name, it can essentially execute any code off the internet with root privileges. I want to restrict this particular role to changing the version of the pre-set image name. – Chris Snell Mar 21 '18 at 19:01
  • @ChrisSnell This is a tough question, I don't have any solution or suggestions on it. The smallest resource that can be authorized by a role is Deployment itself by my opinion. I don't know is there anyway to controll each configuration of the Deployment. – Haoming Zhang Mar 22 '18 at 07:48
3

Markdown version, using kubectl instead of curl

Here follows a different code snippet, derived from the script posted in the answer by John.
When executed in Bash, it produces a more detailed output in the form of a Markdown table, saved as the file Kubernetes_API_resources.md.
It uses kubectl get --raw ... instead of curl to query the API, and the resulting Markdown file documents its own creation in a code block.

echo "# Kubernetes API resources

Updated on `date -I`

\`\`\`bash
${BASH_COMMAND}
\`\`\`

| API name/version | Resource | Verbs | Kind | Namespaced |
| ---------------- | -------- | ----- | ---- | ---------- |
`
for apipath in $(kubectl api-versions | sort | sed '/\//{H;1h;$!d;x}'); do
  version=${apipath#*/}
  api=${apipath%$version}
  api=${api%/}
  prefix="/api${api:+s}/"
  api=${api:-(core)}
  >&2 echo "${prefix}${apipath}: ${api}/${version}"
  kubectl get --raw "${prefix}${apipath}" | jq -r --arg api "${api}/${version}" '.resources | sort_by(.name) | .[]? | "| \($api) | \(.name) | \(.verbs | join(" ")) | \(.kind) | \(if .namespaced then "true" else "false" end) |"'
done
`" > Kubernetes_API_resources.md
tachylatus
  • 113
  • 1
  • 4
2

I wrote a tiny Go utility for this exact purpose. Generates a complete RBAC role with every possible resource & sub-resource on the cluster. You can then prune that back to fit your role's use case.

https://github.com/coopernetes/kube-role-gen

coopernetes
  • 103
  • 6
2

There is a kubectl plugin - rbac-tool that has a new subcommand that outputs the available resource (and subresource) available permissions.

under the hood it uses the Kubernetes dynamic api client to fetch server API resources for all groups.

Fore example:

$kubectl rbac-tool show --for-groups=,apps

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  annotations: null
  creationTimestamp: null
  labels: null
  name: custom-cluster-role
rules:
- apiGroups:
  - ""
  resources:
  - bindings
  verbs:
  - create
- apiGroups:
  - ""
  resources:
  - componentstatuses
  verbs:
  - get
  - list
- apiGroups:
  - ""
  resources:
  - configmaps
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - endpoints
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - limitranges
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - namespaces
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - namespaces/finalize
  verbs:
  - update
- apiGroups:
  - ""
  resources:
  - namespaces/status
  verbs:
  - get
  - patch
  - update
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - nodes/proxy
  verbs:
  - create
  - delete
  - get
  - patch
  - update
- apiGroups:
  - ""
  resources:
  - nodes/status
  verbs:
  - get
  - patch
  - update
- apiGroups:
  - ""
  resources:
  - persistentvolumeclaims
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - persistentvolumeclaims/status
  verbs:
  - get
  - patch
  - update
- apiGroups:
  - ""
  resources:
  - persistentvolumes
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - persistentvolumes/status
  verbs:
  - get
  - patch
  - update
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - pods/attach
  verbs:
  - create
  - get
- apiGroups:
  - ""
  resources:
  - pods/binding
  verbs:
  - create
- apiGroups:
  - ""
  resources:
  - pods/eviction
  verbs:
  - create
- apiGroups:
  - ""
  resources:
  - pods/exec
  verbs:
  - create
  - get
- apiGroups:
  - ""
  resources:
  - pods/log
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - pods/portforward
  verbs:
  - create
  - get
- apiGroups:
  - ""
  resources:
  - pods/proxy
  verbs:
  - create
  - delete
  - get
  - patch
  - update
- apiGroups:
  - ""
  resources:
  - pods/status
  verbs:
  - get
  - patch
  - update
- apiGroups:
  - ""
  resources:
  - podtemplates
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - replicationcontrollers
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - replicationcontrollers/scale
  verbs:
  - get
  - patch
  - update
- apiGroups:
  - ""
  resources:
  - replicationcontrollers/status
  verbs:
  - get
  - patch
  - update
- apiGroups:
  - ""
  resources:
  - resourcequotas
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - resourcequotas/status
  verbs:
  - get
  - patch
  - update
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - serviceaccounts
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - services/proxy
  verbs:
  - create
  - delete
  - get
  - patch
  - update
- apiGroups:
  - ""
  resources:
  - services/status
  verbs:
  - get
  - patch
  - update
- apiGroups:
  - apps
  resources:
  - controllerrevisions
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - apps
  resources:
  - daemonsets
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - apps
  resources:
  - daemonsets/status
  verbs:
  - get
  - patch
  - update
- apiGroups:
  - apps
  resources:
  - deployments
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - apps
  resources:
  - deployments/scale
  verbs:
  - get
  - patch
  - update
- apiGroups:
  - apps
  resources:
  - deployments/status
  verbs:
  - get
  - patch
  - update
- apiGroups:
  - apps
  resources:
  - replicasets
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - apps
  resources:
  - replicasets/scale
  verbs:
  - get
  - patch
  - update
- apiGroups:
  - apps
  resources:
  - replicasets/status
  verbs:
  - get
  - patch
  - update
- apiGroups:
  - apps
  resources:
  - statefulsets
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - apps
  resources:
  - statefulsets/scale
  verbs:
  - get
  - patch
  - update
- apiGroups:
  - apps
  resources:
  - statefulsets/status
  verbs:
  - get
  - patch
  - update
kruzer
  • 21
  • 1
1

run kubectl proxy , the server will start running on http://127.0.0.1:8001/. so just open that in browser, you will see all api-resources

Shiva
  • 11
  • 2
0

If you are using kubectl krew plug-in, I will suggest using get-all. It can get almost 90% resources. included configmap, secret, endpoints, istio, etc

And It have a great arg --since, you can use it to list out last x min created resources.

example 

kubectl get-all --since 1d

enter image description here

Andy Wong
  • 3,676
  • 1
  • 21
  • 18
0

Another option, especially for those who don't have immediate access to a live k8s, is the OpenAPI spec.
From the api reference, you can reach the latest docs which has a link, at the top right, to the git managed OpenAPI spec which you can load at the Swagger live web editor.
Endpoints like /api/v1/namespaces/{namespace}/pods/{name}/log will be listed there.

Placed all these links in an attempt to future-proof this answer. I couldn't find a /latest type URL that'll point to the latest version.

edd
  • 1,307
  • 10
  • 10
0

You can use explain command to get mode details about api-resource and sub resources.

Here I am taking an example of POD api-resource:

kubectl explain pod

KIND:     Pod
VERSION:  v1
DESCRIPTION:
    Pod is a collection of containers that can run on a host. This resource is created by clients and scheduled onto hosts.

If you to want to check more about spec section (sub-resource) of POD, use

kubectl explain pod.spec

For toleration

kubectl explain pod.spec.tolerations

and if you want to get check values and its input type use

kubectl explain pod.spec.tolerations.value

enter image description here

Hope that answers your question

marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
Mayank Tiwari
  • 1