Using Kubernetes Service Accounts from the outside

Service Accounts are the official way to access the Kubernetes API from within pods. The official tutorials on this cover this well, such as the one here.

What this does not explain, is the fact that the Service Account secret can be used from scripts outside a pod, outside the cluster even, such as a script calling kubectl.

Due to the advanced Role Based Access Control (referred to as RBAC) system in Kubernetes, not all Service Accounts are the same. For our access, let us create a fresh one.

Note: you are free to re-use an existing account, but there is no guarantee, as you will see below, that you will find one with the Access Control rights that you need. There are two ways this will go, either the SA that you use will not be able to do what you want, or it may end up with way more rights than needed by the calling script - which from a security standpoint is not ideal, again as per the Principle of Least Privilege.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: myexample-sa
  namespace: myexample

Once we have the Service Account, we need to create a Role that represent the access rights that we would like for our script and bind it to this service account.

First the Role:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: myexample
  name: myexample-role
rules:
- apiGroups: [""] # "" indicates the core API group
  resources: ["pods"]
  verbs: ["get", "watch", "list", "create"]
- apiGroups: [""] # "" indicates the core API group
  resources: ["pods/exec"]
  verbs: ["create"]

You can use ClusterRole instead of Role (to be able to work across namespaces), but we will stick with Role here. There is information here on the details on configuring this CR, but in our example, we wish to allow this Role to use the verbs - get, watch, list and create on all pods.

Note how we have to include the create verb for a resource called pods/exec to provide our script the ability to kubectl exec into pods.

Now, to bind the myexample-role role to the myexample-sa service account:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: myexample-rb
  namespace: myexample
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: myexample-role
subjects:
- kind: ServiceAccount
  name: myexample-sa
  namespace: myexample

Once you create these resources on the cluster, the SA can do what we want.

Now that we have a Service Account, it can be used from pods. We, however, are more interested in the secret token that it holds, so let's extract that out.

First, look at the SA we just created:

$ kubectl -n myexample describe sa myexample-sa

The output will look something like the following:

Name:                myexample-sa
Namespace:           myexample
Labels:              <none>
Annotations:         Image pull secrets:  myexample-sa-dockercfg-xxxxx
Mountable secrets:   myexample-sa-token-xxxxx
                     myexample-sa-dockercfg-xxxxx
Tokens:              myexample-sa-token-xxxxx
                     myexample-sa-token-xxxxx
Events:              <none>

The thing we are interested in is the Tokens. Pick any one of the two available options.

Now, look at the secret it represents:

$ kubectl describe secret myexample-sa-token-xxxxx

It should look something like the following:

Name:         myexample-sa-token-xxxxx
Namespace:    myexample
Labels:       <none>
Annotations:  kubernetes.io/created-by: <...>
              kubernetes.io/service-account.name: myexample-sa
              kubernetes.io/service-account.uid: <...>
Type:  kubernetes.io/service-account-token
Data
====
ca.crt:          5940 bytes
namespace:       9 bytes
service-ca.crt:  8365 bytes
token:
eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJmaXZlZy1jb3JlLWlybCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJmaXZlZ2MtYWNjZXNzLXNhLXRva2VuLWNkcmduIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImZpdmVnYy1hY2Nlc3Mtc2EiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiI3OGUxMDE0OS0wZDIzLTRhNDctYTdmMS00YmFiMzNlMWIxODciLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6Zml2ZWctY29yZS1pcmw6Zml2ZWdjLWFjY2Vzcy1zYSJ9.LnAFgWZdTpzBpiDYQpgqZDwhWBVpqrq97uOzp7qTAWrp825g60uu.9ic9Kz9EqhLUhl_vE-eyJhbGciOiJSUzI1NiIsImtpZCI6IkRhVFYzT3l0UEZrbmN0YkFFUlRfRkhGUUt5TU8tYV80V1Ayc1VXc3pma1kifQ-nSXW3_26sQdXnIA-sTzujFPROvVmWKWJNR0_Y8B2z-MsOH2IGVX9jeiyYByqfACXF83DepVLsJLOQKUbxcXVpjWtKpR4GpWs6pGxiR3ufzMA5BVE2Rgw0e4g_L8zBLDn36SdYFkW_S7wh8-fOKLhuNk1O60afTuZbf4cnzNwZyYgcZBTmrfJQHRlMRS3r2Nz_BXGQWX9VGNVCEvE_vfCxoOxWECeJ6QIbTf9dYcYlde8clVcHxdZKZOmex-m3-oxZKZJdE0vA4QOMBb

The "token" field is what we are interested in. The token shown here of course, is just a randomized string. Note this down.

Note: If you use get -o yaml instead of describe, you will get a base64 encoded version of the token. You will have to manually decode it before you can use it in the next step.

At this point, we have a legitimate API key that can be used to access the Kubernetes API from anywhere. If you were to access the API directly (say from curl), you could use this token as the Bearer token for the Authorization header.

But usually, the Kubernetes clients make you load configs from the KUBECONFIG file (for instance, this official go example), so we need to prepare one, which is what we will do in the next section.

Normally, if you are using multiple clusters, the KUBECONFIG file will be a bit of a mess. It is better to create a new file for this purpose.

The structure of the file should be as follows.

apiVersion: v1
clusters:
- cluster:
    insecure-skip-tls-verify: true
    server: https://<url>:6443
  name: <url>:6443
contexts:
- context:
    cluster: <url>:6443
    namespace: myexample
    user: myexample-sa
  name: default
current-context: default
kind: Config
preferences: {}
users:
- name: myexample-sa
  user:
    token: <your-token>

Let's go over this section by section.

1. First, we have the "clusters" section. You can use the values from your main KUBECONFIG file or the url directly if you know it. There is only one cluster here, but the technique talked about can be repeated any number of times.
2. Second, we have the "context" section. Each context is a namespace and user in a cluster. In our example, we have only one namespace myexample and one user represented by the myexample-sa, but as before, you can repeat the steps above for any number of namespaces and users.
3. The "current-config" is set to whichever of the configs you want to apply by default.
4. In the "users" section, we have a user standing in for our SA. This is the main location of change: you need to fill in the token extracted from the SA secret in the previous section here.

That's it. Once we have this file, running our scripts should be as simple as:

KUBECONFIG=<path to new kubeconfig> myscript

If you have multiple contexts, say for doing things in different namespaces, you can just switch context, for example with something like this inside myscript:

<do something in default context>

kubectl config use-context context1
<do something in default context1>

kubectl config use-context context2
<do something in default context2>

# reset context
kubectl config use-context default