Applying PSPs to istio-cni-node pods

I work on Kubernetes cluster which are a whole lot more locked-down than typical installation instructions / application defaults would suggest. In one such cluster, we use PodSecurityPolicies to apply a minimal set of privileges to each pod, and make exceptions on a case-by-case basis.

On the same cluster, we use the Istio service mesh to secure traffic between our pods using mutualTLS. We take advantage of Istio's CNI plugin to allow the Istio sidecar to inject the "traffic interception" rules when pods start up, without requiring privileged access for every pod with a sidecar.

The CNI plugin creates a daemonset (a pod per node), which requires privileged access to inject the interception rules. Our default, restrictive PSP policy prevents these istio-cni-node pods from ever starting though, as illustrated below:

  Type     Reason        Age                    From                  Message
  ----     ------        ----                   ----                  -------
  Warning  FailedCreate  114s (x17 over 7m22s)  daemonset-controller  Error creating: pods "istio-cni-node-" is forbidden: unable to validate against any pod security policy: [spec.securityContext.hostNetwork: Invalid value: true: Host network is not allowed to be used spec.volumes[0].hostPath.pathPrefix: Invalid value: "/opt/cni/bin": is not allowed to be used spec.volumes[1].hostPath.pathPrefix: Invalid value: "/etc/cni/net.d": is not allowed to be used]

The error above is pointing out that PSPs (quite rightly) prevented an arbitrary pod from mounting critical host directories, and having its way with them.

In this case, access to /opt/cni/bin and /etc/cni/net.d is a requirement for using Istio CNI (and the alternative of allowing every pod privileged access is much worse!), so we deploy a PSP, ClusterRole, and ClusterRoleBinding as illustrated below (you can grab a copy here):