|
3 | 3 | > ## *This page is out-of-date and under active development.* |
4 | 4 |
|
5 | 5 | The |
6 | | -[HostPath](https://github.com/kubernetes-csi/drivers/tree/master/pkg/hostpath) |
7 | | -can be used to provision local storage in a single node test. This |
8 | | -section shows how to deploy and use that driver in Kubernetes. |
| 6 | +[HostPath](https://github.com/kubernetes-csi/csi-driver-host-path) |
| 7 | +can be used to provision local storage. The deployment example there |
| 8 | +shows how to deploy and use that driver in Kubernetes. |
9 | 9 |
|
10 | | -The deployment of a CSI driver determines which RBAC rules are |
11 | | -needed. For example, enabling or disabling leadership election changes |
12 | | -which permissions the external-attacher and external-provisioner need. |
13 | | -This example deployment uses the original RBAC rule files that are |
| 10 | +## Deployment |
| 11 | + |
| 12 | +The example deployment uses the original RBAC rule files that are |
14 | 13 | maintained together with those sidecar apps and deploys into the |
15 | 14 | default namespace. |
16 | 15 |
|
| 16 | +Deployment has been tested with Kubernetes v1.13 |
| 17 | + |
17 | 18 | A real production should copy the RBAC files and customize them as |
18 | 19 | explained in the comments of those files. |
19 | 20 |
|
20 | | - |
21 | | -## Deployment |
22 | | - |
23 | | -This was initially tested with Kubernetes v1.12 and should still work |
24 | | -there. It was also tested with a 1.13 pre-release snapshot. To ensure |
25 | | -that all necessary features are enabled, set the following feature |
26 | | -gate flags to true: |
27 | | - |
28 | | -``` |
29 | | ---feature-gates=CSIPersistentVolume=true,MountPropagation=true,VolumeSnapshotDataSource=true,KubeletPluginsWatcher=true,CSINodeInfo=true,CSIDriverRegistry=true |
30 | | -``` |
31 | | - |
32 | | -`CSIPersistentVolume` is enabled by default in v1.10. `MountPropagation` is enabled by default |
33 | | -in v1.10. `VolumeSnapshotDataSource` is a new alpha feature in v1.12. `KubeletPluginsWatcher` |
34 | | -is enabled by default in v1.12. `CSINodeInfo` and `CSIDriverRegistry` are new alpha features |
35 | | -in v1.12. |
36 | | - |
37 | | -CRDs need to be created manually for `CSIDriverRegistry` and `CSINodeInfo`: |
38 | | - |
39 | | - |
40 | | -> $ kubectl create -f [https://raw.githubusercontent.com/kubernetes/csi-api/ab0df28581235f5350f27ce9c27485850a3b2802/pkg/crd/testdata/csidriver.yaml](https://github.com/kubernetes/csi-api/blob/ab0df28581235f5350f27ce9c27485850a3b2802/pkg/crd/testdata/csidriver.yaml) --validate=false |
41 | | -> `customresourcedefinition.apiextensions.k8s.io/csidrivers.csi.storage.k8s.io created` |
42 | | -> |
43 | | -> $ kubectl create -f [https://raw.githubusercontent.com/kubernetes/csi-api/ab0df28581235f5350f27ce9c27485850a3b2802/pkg/crd/testdata/csinodeinfo.yaml](https://github.com/kubernetes/csi-api/blob/ab0df28581235f5350f27ce9c27485850a3b2802/pkg/crd/testdata/csinodeinfo.yaml) --validate=false |
44 | | -> `customresourcedefinition.apiextensions.k8s.io/csinodeinfos.csi.storage.k8s.io created` |
45 | | -
|
46 | | -### Create RBAC rules for CSI provisioner |
47 | | - |
48 | | -> $ kubectl create -f [https://raw.githubusercontent.com/kubernetes-csi/external-provisioner/1cd1c20a6d4b2fcd25c98a008385b436d61d46a4/deploy/kubernetes/rbac.yaml](https://github.com/kubernetes-csi/external-provisioner/blob/1cd1c20a6d4b2fcd25c98a008385b436d61d46a4/deploy/kubernetes/rbac.yaml) |
49 | | -> ``` |
50 | | -> clusterrole.rbac.authorization.k8s.io/external-provisioner-runner created |
51 | | -> clusterrolebinding.rbac.authorization.k8s.io/csi-provisioner-role created |
52 | | -> role.rbac.authorization.k8s.io/external-provisioner-cfg created |
53 | | -> rolebinding.rbac.authorization.k8s.io/csi-provisioner-role-cfg created |
54 | | -> ``` |
55 | | -
|
56 | | -### Create RBAC rules for CSI attacher |
57 | | -
|
58 | | -> $ kubectl create -f [https://raw.githubusercontent.com/kubernetes-csi/external-attacher/9da8c6d20d58750ee33d61d0faf0946641f50770/deploy/kubernetes/rbac.yaml](https://github.com/kubernetes-csi/external-attacher/blob/9da8c6d20d58750ee33d61d0faf0946641f50770/deploy/kubernetes/rbac.yaml) |
59 | | -> ``` |
60 | | -> serviceaccount/csi-attacher created |
61 | | -> clusterrole.rbac.authorization.k8s.io/external-attacher-runner created |
62 | | -> clusterrolebinding.rbac.authorization.k8s.io/csi-attacher-role created |
63 | | -> role.rbac.authorization.k8s.io/external-attacher-cfg created |
64 | | -> rolebinding.rbac.authorization.k8s.io/csi-attacher-role-cfg created |
65 | | -> ``` |
66 | | -
|
67 | | -### Create RBAC rules for node plugin |
68 | | -
|
69 | | -Only the `driver-registrar` interacts directly with Kubernetes, so it's those RBAC rules that are needed: |
70 | | -
|
71 | | -> $ kubectl create -f [https://raw.githubusercontent.com/kubernetes-csi/driver-registrar/87d0059110a8b4a90a6d2b5a8702dd7f3f270b80/deploy/kubernetes/rbac.yaml](https://github.com/kubernetes-csi/driver-registrar/blob/87d0059110a8b4a90a6d2b5a8702dd7f3f270b80/deploy/kubernetes/rbac.yaml) |
72 | | -> ``` |
73 | | -> serviceaccount/csi-driver-registrar created |
74 | | -> clusterrole.rbac.authorization.k8s.io/driver-registrar-runner created |
75 | | -> clusterrolebinding.rbac.authorization.k8s.io/csi-driver-registrar-role created |
76 | | -> ``` |
77 | | -
|
78 | | -### Create RBAC rules for CSI snapshotter |
79 | | -
|
80 | | -The CSI snapshotter is an optional sidecar container. You only need to create these |
81 | | -RBAC rules if you want to test the snapshot feature. |
82 | | -
|
83 | | -> $ kubectl create -f [https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/01bd7f356e6718dee87914232d287631655bef1d/deploy/kubernetes/rbac.yaml](https://github.com/kubernetes-csi/external-snapshotter/blob/01bd7f356e6718dee87914232d287631655bef1d/deploy/kubernetes/rbac.yaml) |
84 | | -> ``` |
85 | | -> serviceaccount/csi-snapshotter created |
86 | | -> clusterrole.rbac.authorization.k8s.io/external-snapshotter-runner created |
87 | | -> clusterrolebinding.rbac.authorization.k8s.io/csi-snapshotter-role created |
88 | | -> ``` |
89 | | -
|
90 | | -### Deploy driver-registrar and hostpath CSI plugin in DaemonSet pod |
| 21 | +CRDs are created optionally `CSIDriverRegistry` and `CSINodeInfo`: |
91 | 22 |
|
92 | 23 | The CSI sidecar apps are going to connect to the CSI driver, therefore |
93 | 24 | starting it first helps avoid timeouts and intermittent container |
94 | | -restarts: |
95 | | -
|
96 | | -> $ kubectl create -f [https://raw.githubusercontent.com/kubernetes/kubernetes/f40a5d1155aae95105a4e9bb8933d750c666e350/test/e2e/testing-manifests/storage-csi/hostpath/hostpath/csi-hostpathplugin.yaml](https://github.com/kubernetes/kubernetes/blob/f40a5d1155aae95105a4e9bb8933d750c666e350/test/e2e/testing-manifests/storage-csi/hostpath/hostpath/csi-hostpathplugin.yaml) |
97 | | -> `daemonset.apps/csi-hostpathplugin created` |
98 | | -> |
99 | | -> $ kubectl get pod |
100 | | -> ``` |
101 | | -> NAME READY STATUS RESTARTS AGE |
102 | | -> csi-hostpathplugin-4k7hk 2/2 Running 0 22s |
103 | | -> ``` |
104 | | -
|
105 | | -### Deploy CSI provisioner in StatefulSet pod |
106 | | -
|
107 | | -> $ kubectl create -f [https://raw.githubusercontent.com/kubernetes/kubernetes/f40a5d1155aae95105a4e9bb8933d750c666e350/test/e2e/testing-manifests/storage-csi/hostpath/hostpath/csi-hostpath-provisioner.yaml](https://github.com/kubernetes/kubernetes/blob/f40a5d1155aae95105a4e9bb8933d750c666e350/test/e2e/testing-manifests/storage-csi/hostpath/hostpath/csi-hostpath-provisioner.yaml) |
108 | | -> ``` |
109 | | -> service/csi-hostpath-provisioner created |
110 | | -> statefulset.apps/csi-hostpath-provisioner created |
111 | | -> ``` |
112 | | -> |
113 | | -> $ kubectl get pod |
114 | | -> ``` |
115 | | -> NAME READY STATUS RESTARTS AGE |
116 | | -> csi-hostpath-provisioner-0 1/1 Running 0 14s |
117 | | -> csi-hostpathplugin-4k7hk 2/2 Running 0 75s |
118 | | -> ``` |
119 | | -
|
120 | | -
|
121 | | -### Deploy CSI attacher in StatefulSet pod |
122 | | -
|
123 | | -> $ kubectl create -f [https://raw.githubusercontent.com/kubernetes/kubernetes/f40a5d1155aae95105a4e9bb8933d750c666e350/test/e2e/testing-manifests/storage-csi/hostpath/hostpath/csi-hostpath-attacher.yaml](https://github.com/kubernetes/kubernetes/blob/f40a5d1155aae95105a4e9bb8933d750c666e350/test/e2e/testing-manifests/storage-csi/hostpath/hostpath/csi-hostpath-attacher.yaml) |
124 | | -> ``` |
125 | | -> service/csi-hostpath-attacher created |
126 | | -> statefulset.apps/csi-hostpath-attacher created |
127 | | -> ``` |
128 | | -> |
129 | | -> $ kubectl get pod |
130 | | -> ``` |
131 | | -> NAME READY STATUS RESTARTS AGE |
132 | | -> csi-hostpath-attacher-0 1/1 Running 0 14s |
133 | | -> csi-hostpath-provisioner-0 1/1 Running 0 56s |
134 | | -> csi-hostpathplugin-4k7hk 2/2 Running 0 117s |
135 | | -> ``` |
| 25 | +restarts |
136 | 26 |
|
137 | 27 | ### Deploy livenessprobe with CSI plugin |
138 | 28 |
|
@@ -192,177 +82,10 @@ Where: |
192 | 82 |
|
193 | 83 | - `--health-port` - specifies the TCP ports for listening healthz requests (default "9808") |
194 | 84 |
|
195 | | - |
196 | | -### Deploy CSI snapshotter in StatefulSet pod |
197 | | - |
198 | | -The CSI snapshotter is an optional sidecar container. You only need to deploy it if you |
199 | | -want to test the snapshot feature. |
200 | | - |
201 | | -> $ kubectl create -f [https://raw.githubusercontent.com/kubernetes-csi/docs/387dce893e59c1fcf3f4192cbea254440b6f0f07/book/src/example/snapshot/csi-hostpath-snapshotter.yaml](https://github.com/kubernetes-csi/docs/blob/387dce893e59c1fcf3f4192cbea254440b6f0f07/book/src/example/snapshot/csi-hostpath-snapshotter.yaml) |
202 | | -> ``` |
203 | | -> service/csi-hostpath-snapshotter created |
204 | | -> statefulset.apps/csi-hostpath-snapshotter created |
205 | | -> ``` |
206 | | -> |
207 | | -> $ kubectl get pod |
208 | | -> ``` |
209 | | -> NAME READY STATUS RESTARTS AGE |
210 | | -> csi-hostpath-attacher-0 1/1 Running 0 58s |
211 | | -> csi-hostpath-provisioner-0 1/1 Running 0 100s |
212 | | -> csi-hostpath-snapshotter-0 1/1 Running 0 12s |
213 | | -> csi-hostpathplugin-4k7hk 2/2 Running 0 2m41s |
214 | | -> ``` |
215 | | -
|
216 | | -
|
217 | | -## Usage |
218 | | -
|
219 | | -Dynamic provisioning is enabled by creating a `csi-hostpath-sc` storage class. |
220 | | -
|
221 | | -> $ kubectl create -f [https://raw.githubusercontent.com/kubernetes-csi/docs/387dce893e59c1fcf3f4192cbea254440b6f0f07/book/src/example/usage/csi-storageclass.yaml](https://github.com/kubernetes-csi/docs/blob/387dce893e59c1fcf3f4192cbea254440b6f0f07/book/src/example/usage/csi-storageclass.yaml) |
222 | | -> `storageclass.storage.k8s.io/csi-hostpath-sc created` |
223 | | -
|
224 | | -We can use this storage class to create and claim a new volume: |
225 | | -
|
226 | | -> $ kubectl create -f [https://raw.githubusercontent.com/kubernetes-csi/docs/387dce893e59c1fcf3f4192cbea254440b6f0f07/book/src/example/usage/csi-pvc.yaml](https://github.com/kubernetes-csi/docs/blob/387dce893e59c1fcf3f4192cbea254440b6f0f07/book/src/example/usage/csi-pvc.yaml) |
227 | | -> `persistentvolumeclaim/csi-pvc created` |
228 | | -> |
229 | | -> $ kubectl get pvc |
230 | | -> ``` |
231 | | -> NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE |
232 | | -> csi-pvc Bound pvc-0571cc14-c714-11e8-8911-000c2967769a 1Gi RWO csi-hostpath-sc 3s |
233 | | -> ``` |
234 | | -> |
235 | | -> $ kubectl get pv |
236 | | -> ``` |
237 | | -> NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE |
238 | | -> pvc-0571cc14-c714-11e8-8911-000c2967769a 1Gi RWO Delete Bound default/csi-pvc csi-hostpath-sc 3s |
239 | | -> ``` |
240 | | -
|
241 | | -The HostPath driver is configured to create new volumes under `/tmp` inside |
242 | | -the `hostpath` container in the CSI hostpath plugin DaemonSet pod and thus |
243 | | -persist as long as the DaemonSet pod itself. |
244 | | -We can use such volumes in another pod like this: |
245 | | -
|
246 | | -> $ kubectl create -f [https://raw.githubusercontent.com/kubernetes-csi/docs/387dce893e59c1fcf3f4192cbea254440b6f0f07/book/src/example/usage/csi-app.yaml](https://github.com/kubernetes-csi/docs/blob/387dce893e59c1fcf3f4192cbea254440b6f0f07/book/src/example/usage/csi-app.yaml) |
247 | | -> `pod/my-csi-app created` |
248 | | -> |
249 | | -> $ kubectl get pods |
250 | | -> ``` |
251 | | -> NAME READY STATUS RESTARTS AGE |
252 | | -> csi-hostpath-attacher-0 1/1 Running 0 117s |
253 | | -> csi-hostpath-provisioner-0 1/1 Running 0 2m39s |
254 | | -> csi-hostpath-snapshotter-0 1/1 Running 0 71s |
255 | | -> csi-hostpathplugin-4k7hk 2/2 Running 0 3m40s |
256 | | -> my-csi-app 1/1 Running 0 14s |
257 | | -> ``` |
258 | | -> |
259 | | -> $ kubectl describe pods/my-csi-app |
260 | | -> ``` |
261 | | -> Name: my-csi-app |
262 | | -> Namespace: default |
263 | | -> Priority: 0 |
264 | | -> PriorityClassName: <none> |
265 | | -> Node: 127.0.0.1/127.0.0.1 |
266 | | -> Start Time: Wed, 03 Oct 2018 06:59:19 -0700 |
267 | | -> Labels: <none> |
268 | | -> Annotations: <none> |
269 | | -> Status: Running |
270 | | -> IP: 172.17.0.5 |
271 | | -> Containers: |
272 | | -> my-frontend: |
273 | | -> Container ID: docker://fd2950af39a155bdf08d1da341cfb23aa0d1af3eaaad6950a946355789606e8c |
274 | | -> Image: busybox |
275 | | -> Image ID: docker-pullable://busybox@sha256:2a03a6059f21e150ae84b0973863609494aad70f0a80eaeb64bddd8d92465812 |
276 | | -> Port: <none> |
277 | | -> Host Port: <none> |
278 | | -> Command: |
279 | | -> sleep |
280 | | -> 1000000 |
281 | | -> State: Running |
282 | | -> Started: Wed, 03 Oct 2018 06:59:22 -0700 |
283 | | -> Ready: True |
284 | | -> Restart Count: 0 |
285 | | -> Environment: <none> |
286 | | -> Mounts: |
287 | | -> /data from my-csi-volume (rw) |
288 | | -> /var/run/secrets/kubernetes.io/serviceaccount from default-token-xms2g (ro) |
289 | | -> Conditions: |
290 | | -> Type Status |
291 | | -> Initialized True |
292 | | -> Ready True |
293 | | -> ContainersReady True |
294 | | -> PodScheduled True |
295 | | -> Volumes: |
296 | | -> my-csi-volume: |
297 | | -> Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace) |
298 | | -> ClaimName: csi-pvc |
299 | | -> ReadOnly: false |
300 | | -> default-token-xms2g: |
301 | | -> Type: Secret (a volume populated by a Secret) |
302 | | -> SecretName: default-token-xms2g |
303 | | -> Optional: false |
304 | | -> QoS Class: BestEffort |
305 | | -> Node-Selectors: <none> |
306 | | -> Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s |
307 | | -> node.kubernetes.io/unreachable:NoExecute for 300s |
308 | | -> Events: |
309 | | -> Type Reason Age From Message |
310 | | -> ---- ------ ---- ---- ------- |
311 | | -> Normal Scheduled 69s default-scheduler Successfully assigned default/my-csi-app to 127.0.0.1 |
312 | | -> Normal SuccessfulAttachVolume 69s attachdetach-controller AttachVolume.Attach succeeded for volume "pvc-0571cc14-c714-11e8-8911-000c2967769a" |
313 | | -> Normal Pulling 67s kubelet, 127.0.0.1 pulling image "busybox" |
314 | | -> Normal Pulled 67s kubelet, 127.0.0.1 Successfully pulled image "busybox" |
315 | | -> Normal Created 67s kubelet, 127.0.0.1 Created container |
316 | | -> Normal Started 66s kubelet, 127.0.0.1 Started container |
317 | | -> ``` |
318 | | -
|
319 | | -
|
320 | | -## Confirming the setup |
321 | | -
|
322 | | -Writing inside the app container should be visible in `/tmp` of the `hostpath` container: |
323 | | -``` |
324 | | -$ kubectl exec -it my-csi-app /bin/sh |
325 | | -/ # touch /data/hello-world |
326 | | -/ # exit |
327 | | - |
328 | | -$ kubectl exec -it $(kubectl get pods --selector app=csi-hostpathplugin -o jsonpath='{.items[*].metadata.name}') -c hostpath /bin/sh |
329 | | -/ # find / -name hello-world |
330 | | -/tmp/057485ab-c714-11e8-bb16-000c2967769a/hello-world |
331 | | -/ # exit |
332 | | -``` |
333 | | -
|
334 | | -There should be a `VolumeAttachment` while the app has the volume mounted: |
335 | | -
|
336 | | -> $ kubectl get VolumeAttachment |
337 | | -> ``` |
338 | | -> Name: csi-a4e97f3af2161c6d081b8e96c58ed00c9bf1e1745e89b2545e24505437f015df |
339 | | -> Namespace: |
340 | | -> Labels: <none> |
341 | | -> Annotations: <none> |
342 | | -> API Version: storage.k8s.io/v1beta1 |
343 | | -> Kind: VolumeAttachment |
344 | | -> Metadata: |
345 | | -> Creation Timestamp: 2018-10-03T13:59:19Z |
346 | | -> Resource Version: 1730 |
347 | | -> Self Link: /apis/storage.k8s.io/v1beta1/volumeattachments/csi-a4e97f3af2161c6d081b8e96c58ed00c9bf1e1745e89b2545e24505437f015df |
348 | | -> UID: 862d7241-c714-11e8-8911-000c2967769a |
349 | | -> Spec: |
350 | | -> Attacher: csi-hostpath |
351 | | -> Node Name: 127.0.0.1 |
352 | | -> Source: |
353 | | -> Persistent Volume Name: pvc-0571cc14-c714-11e8-8911-000c2967769a |
354 | | -> Status: |
355 | | -> Attached: true |
356 | | -> Events: <none> |
357 | | -> ``` |
358 | | -
|
359 | 85 | ## Snapshot support |
360 | 86 |
|
361 | | -Enable dynamic provisioning of volume snapshot by creating a volume snapshot |
362 | | -class as follows: |
| 87 | +Deployment example starts the snapshotter pod. |
363 | 88 |
|
364 | | -> $ kubectl create -f [https://raw.githubusercontent.com/kubernetes-csi/docs/387dce893e59c1fcf3f4192cbea254440b6f0f07/book/src/example/snapshot/csi-snapshotclass.yaml](https://github.com/kubernetes-csi/docs/blob/387dce893e59c1fcf3f4192cbea254440b6f0f07/book/src/example/snapshot/csi-snapshotclass.yaml) |
365 | | -> `volumesnapshotclass.snapshot.storage.k8s.io/csi-hostpath-snapclass created` |
366 | 89 | > |
367 | 90 | > $ kubectl get volumesnapshotclass |
368 | 91 | > ``` |
|
0 commit comments