Skip to content

Commit 15e99a2

Browse files
PushkarJPushkarJ
committed
Tutorial for pod security admission
1 parent b81dcba commit 15e99a2

File tree

4 files changed

+403
-0
lines changed

4 files changed

+403
-0
lines changed

‎content/en/docs/tutorials/_index.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,11 @@ Before walking through each tutorial, you may want to bookmark the
5757

5858
* [Using Source IP](/docs/tutorials/services/source-ip/)
5959

60+
## Pod Security
61+
62+
* [Applying Pod Security Standards at Cluster level](/docs/tutorials/pod-security/cluster-level-pss/)
63+
* [Applying Pod Security Standards at Namespace level](/docs/tutorials/pod-security/ns-level-pss/)
64+
6065
## {{% heading "whatsnext" %}}
6166

6267
If you would like to write a tutorial, see

‎content/en/docs/tutorials/pod-security/_index.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
title: "Pod Security"
3+
weight: 40
4+
---
5+

‎content/en/docs/tutorials/pod-security/cluster-level-pss.md‎

Lines changed: 271 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,271 @@
1+
---
2+
title: Applying Pod Security Standards at Cluster level
3+
content_type: tutorial
4+
weight: 10
5+
---
6+
7+
{{% alert title="Disclaimer" %}}
8+
This tutorial applies only for new clusters.
9+
{{% /alert %}}
10+
11+
## {{% heading "prerequisites" %}}
12+
13+
Install the following on your workstation:
14+
15+
- [KinD](https://kind.sigs.k8s.io/docs/user/quick-start/#installation)
16+
- [kubectl](https://kubernetes.io/docs/tasks/tools/)
17+
18+
# Finding the right Pod Security Standard
19+
20+
As of
21+
v1.23, [Pod Security Admission](/docs/concepts/security/pod-security-admission/)
22+
allows usage of
23+
built-in [Pod Security Standards](/docs/concepts/security/pod-security-standards/)
24+
that can be set in three different modes: `enforce`, `audit`, `warn`.
25+
26+
Following steps allows gathering information about which Pod Security Standards
27+
will be most appropriate to apply:
28+
29+
1. Create a cluster without any pod security standard applied:
30+
31+
```
32+
$ kind create cluster --name psa-wo-cluster-pss --image kindest/node:latest
33+
Creating cluster "psa-wo-cluster-pss" ...
34+
✓ Ensuring node image (kindest/node:latest) 🖼
35+
✓ Preparing nodes 📦
36+
✓ Writing configuration 📜
37+
✓ Starting control-plane 🕹️
38+
✓ Installing CNI 🔌
39+
✓ Installing StorageClass 💾
40+
Set kubectl context to "kind-psa-wo-cluster-pss"
41+
You can now use your cluster with:
42+
43+
kubectl cluster-info --context kind-psa-wo-cluster-pss
44+
45+
Thanks for using kind! 😊
46+
47+
```
48+
49+
2. Switch kubectl context to point it to this new cluster
50+
```
51+
$ kubectl cluster-info --context kind-psa-wo-cluster-pss
52+
Kubernetes control plane is running at https://127.0.0.1:61350
53+
CoreDNS is running at https://127.0.0.1:61350/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
54+
55+
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
56+
```
57+
58+
3. Let's look at existing namespaces in the cluster:
59+
60+
```
61+
$ kubectl get ns
62+
NAME STATUS AGE
63+
default Active 9m30s
64+
kube-node-lease Active 9m32s
65+
kube-public Active 9m32s
66+
kube-system Active 9m32s
67+
local-path-storage Active 9m26s
68+
```
69+
70+
4. Use `--dry-run=server` to understand what happens when different pod security
71+
standard are applied:
72+
73+
```
74+
$ kubectl label --dry-run=server --overwrite ns --all \
75+
pod-security.kubernetes.io/enforce=privileged
76+
namespace/default labeled
77+
namespace/kube-node-lease labeled
78+
namespace/kube-public labeled
79+
namespace/kube-system labeled
80+
namespace/local-path-storage labeled
81+
82+
83+
$ kubectl label --dry-run=server --overwrite ns --all \
84+
pod-security.kubernetes.io/enforce=baseline
85+
namespace/default labeled
86+
namespace/kube-node-lease labeled
87+
namespace/kube-public labeled
88+
Warning: existing pods in namespace "kube-system" violate the new PodSecurity enforce level "baseline:latest"
89+
Warning: etcd-psa-wo-cluster-pss-control-plane (and 3 other pods): host namespaces, hostPath volumes
90+
Warning: kindnet-vzj42: non-default capabilities, host namespaces, hostPath volumes
91+
Warning: kube-proxy-m6hwf: host namespaces, hostPath volumes, privileged
92+
namespace/kube-system labeled
93+
namespace/local-path-storage labeled
94+
95+
96+
$ kubectl label --dry-run=server --overwrite ns --all \
97+
pod-security.kubernetes.io/enforce=restricted
98+
namespace/default labeled
99+
namespace/kube-node-lease labeled
100+
namespace/kube-public labeled
101+
Warning: existing pods in namespace "kube-system" violate the new PodSecurity enforce level "restricted:latest"
102+
Warning: coredns-7bb9c7b568-hsptc (and 1 other pod): unrestricted capabilities, runAsNonRoot != true, seccompProfile
103+
Warning: etcd-psa-wo-cluster-pss-control-plane (and 3 other pods): host namespaces, hostPath volumes, allowPrivilegeEscalation != false, unrestricted capabilities, restricted volume types, runAsNonRoot != true
104+
Warning: kindnet-vzj42: non-default capabilities, host namespaces, hostPath volumes, allowPrivilegeEscalation != false, unrestricted capabilities, restricted volume types, runAsNonRoot != true, seccompProfile
105+
Warning: kube-proxy-m6hwf: host namespaces, hostPath volumes, privileged, allowPrivilegeEscalation != false, unrestricted capabilities, restricted volume types, runAsNonRoot != true, seccompProfile
106+
namespace/kube-system labeled
107+
Warning: existing pods in namespace "local-path-storage" violate the new PodSecurity enforce level "restricted:latest"
108+
Warning: local-path-provisioner-d6d9f7ffc-lw9lh: allowPrivilegeEscalation != false, unrestricted capabilities, runAsNonRoot != true, seccompProfile
109+
namespace/local-path-storage labeled
110+
```
111+
112+
It seems like `privileged` Pod Security Standard when applied, shows no warnings
113+
for any namespaces. However, `baseline` and `restricted` standards both have
114+
more warnings specifically on `kube-system` namespace.
115+
116+
# Setting modes, versions and standards
117+
118+
For the purposes of this tutorial, `baseline` standard will be `enforced` as per
119+
`latest` version whereas `restricted` standard will be applied in
120+
`warn` and `audit` mode. Baseline Pod Security Standard provides a convenient
121+
middle ground that allows keeping the exemption list short and prevents known
122+
privilege escalations.
123+
124+
However, to prevent pods from failing in `kube-system`, this namespace will be
125+
exempt from the cluster level Pod Security Standards applied.
126+
127+
_Notes_:
128+
129+
1. Based on risk posture applied to a cluster, a more strict Pod Security
130+
Standard like `restricted` could be a better choice.
131+
2. This exemption for `kube-system` namespace, allows pods to run as
132+
`privileged` in this namespace. So, it is recommended to apply strict RBAC
133+
policies that limit access to `kube-system` following least privilege
134+
principle.
135+
136+
Let's create a configuration file that can be consumed by built-in Pod Security
137+
Admission Controller to implement these Pod Security Standards:
138+
139+
```
140+
$ mkdir -p /tmp/pss
141+
$ cat <<EOF > /tmp/pss/cluster-level-pss.yaml
142+
apiVersion: apiserver.config.k8s.io/v1
143+
kind: AdmissionConfiguration
144+
plugins:
145+
- name: PodSecurity
146+
configuration:
147+
apiVersion: pod-security.admission.config.k8s.io/v1beta1
148+
kind: PodSecurityConfiguration
149+
defaults:
150+
enforce: "baseline"
151+
enforce-version: "latest"
152+
audit: "restricted"
153+
audit-version: "latest"
154+
warn: "restricted"
155+
warn-version: "latest"
156+
exemptions:
157+
usernames: []
158+
runtimeClasses: []
159+
namespaces: [kube-system]
160+
EOF
161+
```
162+
163+
# Configure the API server
164+
165+
Let's configure API server to consume this file during cluster creation:
166+
167+
```
168+
$ cat <<EOF > /tmp/pss/cluster-config.yaml
169+
kind: Cluster
170+
apiVersion: kind.x-k8s.io/v1alpha4
171+
nodes:
172+
- role: control-plane
173+
kubeadmConfigPatches:
174+
- |
175+
kind: ClusterConfiguration
176+
apiServer:
177+
extraArgs:
178+
admission-control-config-file: /etc/config/cluster-level-pss.yaml
179+
extraVolumes:
180+
- name: accf
181+
hostPath: /etc/config
182+
mountPath: /etc/config
183+
readOnly: false
184+
pathType: "DirectoryOrCreate"
185+
extraMounts:
186+
- hostPath: /tmp/pss
187+
containerPath: /etc/config
188+
# optional: if set, the mount is read-only.
189+
# default false
190+
readOnly: false
191+
# optional: if set, the mount needs SELinux relabeling.
192+
# default false
193+
selinuxRelabel: false
194+
# optional: set propagation mode (None, HostToContainer or Bidirectional)
195+
# see https://kubernetes.io/docs/concepts/storage/volumes/#mount-propagation
196+
# default None
197+
propagation: None
198+
EOF
199+
```
200+
201+
_Note_: Please ensure that when using Docker Desktop with KinD, the `/tmp`
202+
directory is added as a Shared Directory under
203+
`Preferences -> Resources -> File Sharing` (as seen on Docker Desktop for Mac)
204+
205+
# Create cluster with Pod Security Standards
206+
207+
Now we are ready to create a cluster that consumes these pod security standards
208+
configured with pod security admission controller:
209+
210+
```
211+
$ kind create cluster --name psa-with-cluster-pss --image kindest/node:latest --config /tmp/pss/cluster-config.yaml
212+
Creating cluster "psa-with-cluster-pss" ...
213+
✓ Ensuring node image (kindest/node:latest) 🖼
214+
✓ Preparing nodes 📦
215+
✓ Writing configuration 📜
216+
✓ Starting control-plane 🕹️
217+
✓ Installing CNI 🔌
218+
✓ Installing StorageClass 💾
219+
Set kubectl context to "kind-psa-with-cluster-pss"
220+
You can now use your cluster with:
221+
222+
kubectl cluster-info --context kind-psa-with-cluster-pss
223+
224+
Have a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community 🙂
225+
226+
227+
$ kubectl cluster-info --context kind-psa-with-cluster-pss
228+
Kubernetes control plane is running at https://127.0.0.1:63855
229+
CoreDNS is running at https://127.0.0.1:63855/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
230+
231+
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
232+
```
233+
234+
# Create pod
235+
236+
Creating a pod with minimal configuration in default namespace should now be
237+
successful with warnings for `restricted` pod security standards:
238+
239+
```
240+
$ cat <<EOF > /tmp/pss/nginx-pod.yaml
241+
apiVersion: v1
242+
kind: Pod
243+
metadata:
244+
name: nginx
245+
spec:
246+
containers:
247+
- image: nginx
248+
name: nginx
249+
ports:
250+
- containerPort: 80
251+
EOF
252+
253+
254+
$ kubectl apply -f /tmp/pss/nginx-pod.yaml
255+
Warning: would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "nginx" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "nginx" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "nginx" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "nginx" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
256+
pod/nginx created
257+
```
258+
259+
# Bringing it all together
260+
261+
Running the commands in this
262+
[gist](https://gist.github.com/PushkarJ/9f7a0045f4bec31097bdd1e9db0f2f6e)
263+
that were discussed in this tutorial will do all these steps in one go:
264+
265+
1. Create a Pod Security Standards based cluster level Configuration
266+
2. Create a file to let API server consumes this configuration
267+
3. Create a cluster that creates an API server with this configuration
268+
4. Set kubectl context to this new cluster
269+
5. Create a minimal pod yaml file
270+
6. Apply this file to create a Pod in the new cluster
271+

0 commit comments

Comments
 (0)