Enforcing policies with Kubewarden on Amazon EKS

According to Red Hat’s 2022 State of Kubernetes Security Report, respondents stated that exposures due to misconfigurations in their container and Kubernetes environments (46%) is nearly three times the level of concern over attacks (16%), with vulnerabilities as the second-leading cause of worry (28%). Important settings, such as role-based access control (RBAC) and security contexts, are critical to the security posture of a cluster. One of the most important mis configuration is pod that lacks correct security configurations. Kubernetes offered Pod Security Policy (PSP) mechanism to regulate pod security. PSPs define a set of security parameters that pods must fulfil before being created or updated in a cluster. However, PSPs have been deprecated as of Kubernetes version 1.21 due to serious usability problems, and have been removed in Kubernetes version 1.25.

PSPs are decommissioned in favor of Pod Security Admission (PSA), a built-in admission controller that implements the security measures described in the Pod Security Standards (PSS). There are also several Policy-as-Code (PaC) solutions available for Kubernetes that are more flexible than PSS and also provide guardrails to guide cluster users, prevent unwanted behaviors, through prescribed and automated controls. Following are some of the examples

PaC Solution	Link	CNCF Project Status
Open Policy Agent	https://www.openpolicyagent.org/	Graduated
OPA/gatekeeper	https://github.com/open-policy-agent/gatekeeper	Graduated
Kyverno	https://kyverno.io/	Incubating
Kubewarden	https://www.kubewarden.io/	Sandbox

PaC solutions uses Kubernetes Dynamic Admission Controllers to intercept the Kubernetes API server request flow, via a webhook call, and mutate and validate request payloads, based on policies written and stored as code. Mutation and validation happen before the API server request results in a change to the cluster.

To implement Pod security, Kubernetes users can choose between PSA or PaC, both solutions can coexist with PSP in the same cluster. Considering PaC solutions are more flexible and more granular and it is not just focused on pods but can also be used against different resources and actions. It can further be used to implement behaviors that are not necessarily security related, such as best practices, organizational standards, etc.

In this post, we are going to look at PaC solution – Kubewarden, a policy engine for Kubernetes. It doesn’t require users to learn new Domain Specific Language or a query language instead policies can be authored in your favorite programming language. Kubewarden policies can also be distributed using container registries and could be integrated to your existing infrastructure and processes via CI/CD pipelines.

Use Case

Kubewarden installs as an Kubernetes Dynamic Admission Controller, which receives webhook events when an API object changes. It validates incoming requests using policies written in WebAssembly. By using WebAssembly, users can write Kubernetes policies using their favorite programming language, as long as the language can produce Wasm binaries.

Kubewarden has three main components which you will interact with:

1. PolicyServer – component which executes the Kubewarden policies when requests arrive and validates them.
2. ClusterAdmissionPolicy – defines how policies evaluate requests.
3. AdmissionPolicy – policy will process only the requests that are targeting the Namespace

We will show you in this article how Kubernetes cluster administrators can validate and mutate configurations.

Prerequisites

We will assume that you already have an EKS cluster up and running. If you don’t have the cluster, please refer this link to get started with Amazon EKS. Please note: Your k8s cluster version must be above v1.14.

Install Kubewarden on your cluster

Kubewarden installation is easy and the output from the install process is as shown below.

Install cert-manager

kubectl apply -f https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml

Check if cert-manager is up and running

kubectl wait --for=condition=Available deployment --timeout=2m -n cert-manager –all

Deploy Kubewarden stack using helm charts

helm repo add kubewarden https://charts.kubewarden.io

 

helm install --wait -n kubewarden --create-namespace kubewarden-crds kubewarden/kubewarden-crds

helm install --wait -n kubewarden kubewarden-controller kubewarden/kubewarden-controller

helm install --wait -n kubewarden kubewarden-defaults kubewarden/kubewarden-defaults

Now that we have deployed Kubewarden, lets enforce our first policy in the next section.

Demonstration on validating privileged containers

The use of privileged containers is not a good security practice. By using privileged containers, it gives the container access to all of the capabilities that a host has. Compromised containers with privileged access can affect other containers operating on the host as well. In this demonstration, we will show you how to establish a policy that prevents pods that require privileged capabilities from running.

First, let’s create the policy below using kubectl apply command as below. This policy below will not allow a process to run in a privileged mode.

kubectl apply -f - <<EOF
apiVersion: policies.kubewarden.io/v1alpha2
kind: ClusterAdmissionPolicy
metadata:
  name: psp-allowprivilegeescalation
spec:
  module: registry://ghcr.io/kubewarden/policies/allow-privilege-escalation-psp:v0.1.11
  rules:
    - apiGroups:
        - ""
      apiVersions:
        - v1
      resources:
        - pods
      operations:
        - CREATE
        - UPDATE
  mutating: false
  settings:
    default_allow_privilege_escalation: false
EOF

Now that policy has been created, let’s create a NGINX pod with privileged access, as shown in the YAML block below.

kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: nginx
    image: nginx
    securityContext:
      allowPrivilegeEscalation: true
  - name: sidecar
    image: sidecar
EOF

As shown above, the pod creation fails as it contains the allowPrivilegeEscalation: true in the above code snippet. This is one of the best security practices where privilege escalations on pods should not be allowed.

Let’s move to next demo which is on blocking pods running as root

Demonstration on blocking pods running as root

Running containers as ‘root’ is not good security practice. This gives the container access to all of the capabilities that a host has and compromised containers with root access can affect other containers operating on the host. In this demonstration, we will show you how to establish a policy that prevents pods that require “root” capabilities from running.

First, let’s create the policy below using kubectl apply command as below. This policy below will not allow containers to run in root mode.

kubectl apply -f - <<EOF
apiVersion: policies.kubewarden.io/v1alpha2
kind: ClusterAdmissionPolicy
metadata:
  name: psp-usergroup
spec:
  module: registry://ghcr.io/kubewarden/policies/user-group-psp:v0.2.0
  rules:
    - apiGroups:
        - ""
      apiVersions:
        - v1
      resources:
        - pods
      operations:
        - CREATE
        - UPDATE
  mutating: true
  settings:
    run_as_user:
      rule: MustRunAsNonRoot
    supplemental_groups:
      rule: MustRunAs
      ranges:
        - min: 1000
          max: 65535
EOF

Now that policy has been created, let’s create a NGINX pod running as root user, as shown in the YAML block below.

kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: nginx
    image: nginx
    securityContext:
      runAsNonRoot: false
      runAsUser: 0
EOF

As shown above, the pod creation fails as it contains the runAsUser:0 in the above code snippet. This is one of the best security practices where running pods as root should not be allowed.

Now that we have seen how to block root access, let’s go to next demo on allowing pod to use the specific port#

Demonstration on allowing pod to use the port 443 only

Using admission controllers, a Kubewarden rule can be enforced to use specific port only. Below sample policy requires applications to run only on 443 port.

kubectl apply -f - <<EOF
apiVersion: policies.kubewarden.io/v1alpha2
kind: ClusterAdmissionPolicy
metadata:
  name: psp-hostnamespaces
spec:
  module: registry://ghcr.io/kubewarden/policies/host-namespaces-psp:v0.1.2
  rules:
    - apiGroups:
        - ""
      apiVersions:
        - v1
      resources:
        - pods
      operations:
        - CREATE
        - UPDATE
  mutating: false
  settings:
    allow_host_ipc: false
    allow_host_pid: false
    allow_host_ports:
      - min: 443
        max: 443
    allow_host_network: false
EOF

Let’s create the following policy using kubectl apply command

Now that policy has been created, let’s create a NGINX pod running on port# 80, as shown in the YAML block below.

kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: nginx
    image: nginx
    imagePullPolicy: IfNotPresent
    ports:
      - containerPort: 80
        hostPort: 80
  - name: sleeping-sidecar
    image: alpine
    command: ["sleep", "1h"]
EOF

As shown above, the pod creation fails as it is running on port# 80. You can customize this policy to allow other or different ports that you trust.

Cleanup

Remove the remove the Kubewarden resources created by uninstalling the helm charts as follow:

helm uninstall --namespace kubewarden kubewarden-defaults

helm uninstall --namespace kubewarden kubewarden-controller

helm uninstall --namespace kubewarden kubewarden-crds

Once the helm charts have been uninstalled, you can remove the Kubernetes namespace that was used to deploy the Kubewarden stack:

kubectl delete namespace kubewarden

Conclusion

PaC solutions such as Kubewarden makes it simple and easy to do policy management on your EKS cluster, more flexible than PSS and also provide guardrails to guide cluster users, prevent unwanted behaviors, through prescribed and automated controls.

References

https://docs.kubewarden.io

https://github.com/kubewarden/