--- key: "kubernetes-workload-security" title: "Run Kubernetes workloads with only the privileges they need" url: "https://softwaresecurity.com/domains/runtime/containers/kubernetes-workload-security/" api: "https://softwaresecurity.com/api/v1/articles/kubernetes-workload-security.json" domain: "runtime" topic: "Runtime, cloud \u0026 APIs" type: "Guide" identifiers: ["CWE-250","A02:2025"] tags: ["Access control","Workload identity","Kubernetes and containers"] stages: ["build","operate"] roles: ["developer","platform-sre"] license: "CC-BY-4.0" license_url: "https://creativecommons.org/licenses/by/4.0/" code_license: "MIT" --- # Run Kubernetes workloads with only the privileges they need Review pod settings, service accounts and effective permissions together before treating a namespace as an isolation boundary. Canonical URL: https://softwaresecurity.com/domains/runtime/containers/kubernetes-workload-security/ ## In short Label each application namespace to enforce the Restricted Pod Security Standard, then make every pod pass it: run as a non-root UID, set `allowPrivilegeEscalation: false`, drop all capabilities, use the `RuntimeDefault` seccomp profile, and add a read-only root filesystem with a small writable volume. Give the pod its own service account bound to a Role that names exactly the resources and verbs it uses. Prove the result with `kubectl auth can-i` and checks inside the running container. - Pod Security Standards define three levels: Privileged, Baseline and Restricted; Pod Security Admission enforces them per namespace. - Restricted accepts `runAsNonRoot` at pod level but requires `allowPrivilegeEscalation: false` and `drop: ["ALL"]` on every container, init and ephemeral containers included. - A non-root, read-only container with a wildcard Role is still an overpowered API client; review RBAC and the pod spec together. - `resourceNames` limits a Role to named objects; it cannot limit `create`, and `list` or `watch` then succeed only with a matching `metadata.name` field selector. - Check the admitted pod, not the manifest: admission can mutate it, and only the running process shows the real UID and capabilities. ## Recommended defaults | Setting | Value | Note | Source | | --- | --- | --- | --- | | Namespace label `pod-security.kubernetes.io/enforce` | `restricted`, with `enforce-version` pinned to the tested minor version | — | Kubernetes Pod Security Admission | | `runAsNonRoot` | `true`, with a numeric `runAsUser` the image supports | — | Kubernetes Pod Security Standards | | `allowPrivilegeEscalation` | `false` on every container | — | Kubernetes Pod Security Standards | | `capabilities` | `drop: ["ALL"]`; add back only `NET_BIND_SERVICE` if needed | — | Kubernetes Pod Security Standards | | `seccompProfile.type` | `RuntimeDefault` (or `Localhost`) | — | Kubernetes Pod Security Standards | | `readOnlyRootFilesystem` | `true`, with an `emptyDir` that has a `sizeLimit` for scratch files | Recommended; not required by Restricted. | Kubernetes security checklist | | `automountServiceAccountToken` | `false` when the application never calls the Kubernetes API | — | Kubernetes RBAC good practices | ## Verify it 1. Run `kubectl auth can-i get configmap/status-settings -n demo --as=system:serviceaccount:demo:status` and expect yes; expect no for `list configmaps`, `get secret/...` and `create pods`. 2. Run `kubectl exec` with `id` and `grep CapEff /proc/1/status` and expect UID 10001 and `CapEff: 0000000000000000`. 3. Write to `/tmp` and to `/etc` inside the container: the first must succeed and the second must fail with "Read-only file system". 4. Apply a pod with a privileged init container with `--dry-run=server` and expect "violates PodSecurity restricted". 5. Run `kubectl label --dry-run=server --overwrite ns demo pod-security.kubernetes.io/enforce=restricted` to list existing pods that would violate the level before enforcing it. ## What you will learn - Separate Linux process privileges from Kubernetes API permissions - Explain each hardening field using a concrete workload requirement - Test effective authority without breaking necessary application behavior Audience: Developers and platform engineers who write Kubernetes manifests and RBAC rules for their own services Before you start: - Read a Pod specification, security context, service account and namespace. Time: About 27 minutes: 7 to read and 20 for the exercise ## A pod has operating-system and API privileges A workload can reach beyond its intended task through Linux capabilities, host mounts, privileged execution or a powerful Kubernetes identity. Reducing one surface does not automatically reduce the others. Review the pod specification, node/runtime behavior and service-account permissions as one deployment boundary. Pin the Kubernetes version and runtime used for the assessment. The Pod Security Standards define three policy levels: Privileged, which is unrestricted, Baseline, which blocks known privilege escalations, and Restricted, which follows current hardening practice. Pod Security Admission applies a level per namespace through labels, in `enforce`, `audit` or `warn` mode; setting a namespace to `privileged` turns enforcement off for it. An admitted pod is not proof of complete tenant isolation: shared kernels, network access, secrets and administrator powers still matter. Higher-risk mutually untrusted workloads may need stronger runtime or cluster separation, such as the sandboxed node pools in the managed Kubernetes hardening guide at /domains/runtime/containers/managed-kubernetes-hardening/. ## Worked design: a status service that reads one ConfigMap The status process listens on port 8080, reads ConfigMap `status-settings` in namespace `demo` through the Kubernetes API, and writes a bounded cache under `/tmp`. It needs no host files, Secret reads, workload creation or cluster administration. Treat these as explicit requirements. A non-root container with a wildcard service-account role is still an overpowered API client; a narrow role does not make a privileged host-mounted process safe. The manifest below sets a non-root numeric UID, prevents privilege escalation, drops all capabilities, uses the runtime's default seccomp profile and makes the root filesystem read-only. The `/tmp` `emptyDir` supplies the one legitimate writable location with a size limit. The UID must work with the actual image and volume ownership; otherwise the repair produces a crashing service. Do not restore root to silence a permission error before locating the file or directory the process needs. The separate Role grants `get` only for the named ConfigMap. A client implemented as get-by-name can use this. A client that lists or watches every ConfigMap will fail, which is an implementation-policy mismatch to resolve deliberately. A namespaced Role does not authorize another namespace; the RoleBinding must name the dedicated status service account in demo. Because this workload actually calls the API, removing its token mount without providing a supported credential path would break it. The namespace labels enforce the Restricted standard pinned to v1.37 and also warn and audit at the same level. On a Kubernetes 1.37 cluster this manifest, with a test image, was admitted and ran as UID 10001 with no effective capabilities; a write to `/tmp` succeeded and a write to `/etc` failed. Review image provenance, node and runtime configuration separately, and record the admitted pod, because an admission controller can mutate the submitted spec. A manifest linter establishes properties of a document, not the behavior of the running workload. ```yaml apiVersion: v1 kind: Namespace metadata: name: demo labels: # Reject pods that violate the Restricted standard; pin the version you tested pod-security.kubernetes.io/enforce: restricted pod-security.kubernetes.io/enforce-version: v1.37 pod-security.kubernetes.io/warn: restricted pod-security.kubernetes.io/audit: restricted --- apiVersion: v1 kind: ServiceAccount metadata: name: status namespace: demo --- apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: read-status-settings namespace: demo rules: - apiGroups: [""] resources: ["configmaps"] resourceNames: ["status-settings"] verbs: ["get"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: status-reads-settings namespace: demo subjects: - kind: ServiceAccount name: status namespace: demo roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: read-status-settings --- apiVersion: apps/v1 kind: Deployment metadata: name: status namespace: demo spec: replicas: 1 selector: matchLabels: {app: status} template: metadata: labels: {app: status} spec: serviceAccountName: status securityContext: runAsNonRoot: true runAsUser: 10001 runAsGroup: 10001 seccompProfile: {type: RuntimeDefault} containers: - name: status image: registry.example/status@sha256:IMAGE_DIGEST ports: [{containerPort: 8080}] securityContext: allowPrivilegeEscalation: false readOnlyRootFilesystem: true capabilities: {drop: ["ALL"]} resources: requests: {cpu: 50m, memory: 64Mi} limits: {memory: 128Mi} volumeMounts: [{name: scratch, mountPath: /tmp}] volumes: - name: scratch emptyDir: {sizeLimit: 32Mi} ``` ## Choose explicit restrictions and necessary exceptions For a Linux application that supports it, run as a non-root user, prevent privilege escalation, drop unnecessary capabilities and select an appropriate seccomp profile. Avoid privileged containers, host namespaces and `hostPath` mounts unless the workload's actual role requires them. Apply the review to init and ephemeral containers as well as the main application container. A read-only root filesystem can reduce writable executable locations, but provide bounded writable volumes for legitimate temporary work. Set memory, CPU and storage expectations and test failure behavior under pressure. Constraints are useful only if the program starts, writes to its intended locations and handles limits safely; do not solve a deployment failure by granting broad privilege without examining the cause. ## Enforce the Restricted standard at admission Pod Security Admission is built into the API server, so the namespace labels in the manifest are the whole configuration. In the test cluster, a pod whose init container set `privileged: true` was rejected before it was stored, with the message `violates PodSecurity "restricted:v1.37"` listing three violations: the privileged flag, a missing `allowPrivilegeEscalation: false` and capabilities not dropped. Restricted checks init and ephemeral containers as well as application containers. Roll the level out without breaking running workloads: a server-side dry run of the label change lists the existing pods that would violate it, and `warn` and `audit` modes report violations without blocking. Pin `enforce-version` so that a cluster upgrade cannot tighten or loosen the policy without review. The privileged container lab at /labs/kubernetes-policy/ requires every container to declare `runAsNonRoot` itself; Restricted also accepts the pod-level setting used in this manifest, so the lab rule is deliberately stricter than the standard. ## Treat the service account as an application credential Give each workload the Kubernetes API access it needs, scoped to appropriate resources, verbs and namespaces. Avoid wildcard roles and unnecessary token mounting. If the application never calls the API, disable automount where appropriate. Review roles that can read secrets, create workloads, impersonate identities or modify bindings because they can provide indirect access beyond an obvious resource list. Separate the application's cloud identity from its Kubernetes RBAC permissions while reviewing how the two are bound. Protect credentials consumed by the pod and verify rotation behavior; the runtime secrets guide at /domains/identity-crypto/secrets/secrets-in-kubernetes-and-runtime/ covers how. A deployment controller with broad rights may be necessary operationally, but its service account should not be reused as the ordinary application's identity. ## Test the effective deployment Inspect the rendered manifest and admitted pod, then test authorized and forbidden API calls using the workload identity. Confirm actual user, capabilities, mounts and writable locations inside the running container. A client-side policy check can miss admission mutations or cluster settings. Retain exceptions with a workload-specific reason and a test that demonstrates why the narrower configuration cannot yet work. The commands below produced the expected result on each line against the test cluster; images without a shell need `kubectl debug` instead of `exec`. ```bash # API authority of the workload identity (expected answer at the end of each line) SA=system:serviceaccount:demo:status kubectl auth can-i get configmap/status-settings -n demo --as=$SA # yes kubectl auth can-i list configmaps -n demo --as=$SA # no kubectl auth can-i get secret/database-password -n demo --as=$SA # no kubectl auth can-i create pods -n demo --as=$SA # no # Process identity, capabilities and writable paths in the running pod POD=$(kubectl -n demo get pod -l app=status -o name) kubectl -n demo exec "$POD" -- id # uid=10001 gid=10001 kubectl -n demo exec "$POD" -- grep CapEff /proc/1/status # CapEff: 0000000000000000 kubectl -n demo exec "$POD" -- sh -c 'echo ok > /tmp/cache' # succeeds kubectl -n demo exec "$POD" -- sh -c 'echo x > /etc/new-config' # Read-only file system # Admission: a pod with a privileged init container must be rejected kubectl apply --dry-run=server -f bad-init.yaml # violates PodSecurity "restricted:v1.37" ``` ## Diagnose incomplete fixes `runAsNonRoot`, `readOnlyRootFilesystem` and `allowPrivilegeEscalation` protect different things. A read-only root does not prevent reading a mounted secret or writing to a separate writable volume. Preventing privilege escalation does not remove privileges the process already holds. seccomp constrains system calls under the selected profile but is not an authorization policy for cloud APIs. The strongest RBAC review follows indirect authority. Permission to create a pod under a powerful service account, bind roles, impersonate identities or read credentials can exceed what the obvious verb suggests. Prefer a dedicated service account and review the deployer separately. If the application never calls the API, disabling automatic token mounting reduces exposure; that advice does not apply blindly to this API-reading example. ## Check the result against concrete cases These outcomes belong to the status example, and the verification commands above observed each of them on Kubernetes 1.37; a different cluster needs its own run. | Case | Expected result | What it establishes | | --- | --- | --- | | GET demo/status-settings ConfigMap | Allow for bound status service account | Required API use | | LIST all ConfigMaps | Deny in this example | Named-object policy is not a list grant | | GET demo/database-password Secret | Deny | No Secret read authority | | Write `/tmp/cache` | Allow within volume limits | Legitimate process behavior survives | | Write `/etc/new-config` | Fail on read-only root | Root filesystem is constrained | | Init container requests `privileged=true` | Reject under enforced policy | All container lists are covered | ## Guided practice Repair a status workload using the supplied requirements and compare its observed behavior with the desired boundary. The exercise is a manifest and access-decision review, requiring no cluster. ### Supplied material ```text Requirements: GET only demo/status-settings; HTTP 8080; temporary writes under /tmp; no host or Secret access. Original: privileged=true, UID=0, hostPath / mounted at /host, default service account, resources=* verbs=*. Patch A: UID=10001 and readOnlyRootFilesystem=true; all other original settings retained; no scratch volume. Patch B: fragments above; dedicated status service account and exact RoleBinding. Observed under B: startup tries writing /var/cache/status; then API client tries LIST ConfigMaps. Alternative implementation: write /tmp/status; use GET demo/status-settings. ``` ### 1. Explain why A remains unsafe Name two excessive authority paths that survive `UID=10001`, then identify the new availability problem. Worked answer: The wildcard Kubernetes role remains, and privileged execution plus the host mount still expose dangerous host capabilities. A read-only root without the required writable scratch path can break startup. A changed UID alone does not meet the workload contract. ### 2. Map every field to a requirement For B, connect the scratch volume, capability drop and `resourceNames` restriction to the requirement they serve. Worked answer: Scratch allows bounded temporary writes without making the root writable; dropping capabilities reduces unnecessary Linux privileges; `resourceNames` plus `get` narrows Kubernetes reads to `status-settings`. These are independent controls, so one successful check cannot stand in for all three. ### 3. Diagnose the two B failures Decide whether to widen access when startup writes `/var/cache/status` and the client lists ConfigMaps. Worked answer: The supplied requirements support neither behavior. Configure the cache path as `/tmp/status` and the client to `get` the known ConfigMap. If the real product truly needs another operation, revise the requirement and assess its narrow permission; do not restore wildcard rights as the default repair. ### 4. Build a meaningful acceptance record State expected results for the alternative implementation, Secret access, `/etc` writes and a privileged init container. Worked answer: Normal startup and the named ConfigMap read must work; Secret reads and root filesystem writes must fail; an enforced privilege rule must reject the init container too. A service that simply fails every operation has not passed the exercise. Deliverable: A repaired status workload design, a field-to-requirement map, decisions for the two B failures and an acceptance record with allowed and denied cases. Scope: The exercise is a manifest and access-decision review on supplied facts; the verification commands on this page show how to confirm the same outcomes on a real cluster. ## Executable practice - [Privileged container lab: Reject a privileged workload before deployment](https://softwaresecurity.com/labs/kubernetes-policy/) - [NetworkPolicy lab: Lock down a shop namespace](https://softwaresecurity.com/labs/network-policy/) - [Verify Kubernetes admission and network boundaries: project ZIP](https://softwaresecurity.com/downloads/projects/kubernetes-enforcement-6d129f5f9f250337.zip) — Node.js 22+ offline · kind/Docker/kubectl for live cluster Articles and teaching material: CC BY 4.0. Original executable code: MIT. Attribution: Software Security. ## Sources - [Kubernetes Pod Security Standards](https://kubernetes.io/docs/concepts/security/pod-security-standards/). Kubernetes v1.37. - [Kubernetes RBAC good practices](https://kubernetes.io/docs/concepts/security/rbac-good-practices/). Kubernetes v1.37. - [Kubernetes security checklist](https://kubernetes.io/docs/concepts/security/security-checklist/). Kubernetes v1.37. - [Kubernetes Pod Security Admission](https://kubernetes.io/docs/concepts/security/pod-security-admission/). Kubernetes v1.37. - [Kubernetes: enforce Pod Security Standards with namespace labels](https://kubernetes.io/docs/tasks/configure-pod-container/enforce-standards-namespace-labels/). Kubernetes v1.37. - [kubectl auth can-i reference](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_auth/kubectl_auth_can-i/). Kubernetes v1.37.