Add document for Provision volumes from cross namespace data sources #37802

ttakahashi21 · 2022-11-09T02:30:46Z

Add document for Provision volumes from cross namespace data sources

■KEP
Please see CrossNamespaceVolumeDataSource

■API Change (Merge)
kubernetes/kubernetes#113186

■Controller (Under Review)
kubernetes-csi/external-provisioner#805

netlify · 2022-11-09T02:30:59Z

👷 Deploy Preview for kubernetes-io-vnext-staging processing.

Name	Link
🔨 Latest commit	`e837f0f`
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-io-vnext-staging/deploys/6384b02d18486600080cdef8

sftim

Thanks. Please also update https://kubernetes.io/docs/concepts/storage/volume-snapshots/#provisioning-volumes-from-snapshots to document the new optional / alpha feature.

sftim · 2022-11-09T12:15:34Z

content/en/docs/reference/command-line-tools-reference/feature-gates.md

  This feature gate guards *a group* of CPUManager options whose quality level is beta.
  This feature gate will never graduate to stable.
 - `CPUManagerPolicyOptions`: Allow fine-tuning of CPUManager policies.
+- `CrossNamespaceVolumeDataSource`: Enable usage of Provision of PVCs from VolumeSnasphot in other namespaces.


Suggested change

- `CrossNamespaceVolumeDataSource`: Enable usage of Provision of PVCs from VolumeSnasphot in other namespaces.

- `CrossNamespaceVolumeDataSource`: Enable provisioning a {{< glossary_tooltip text="PersistentVolumeClaim" term_id="persistent-volume-claim" >}}

(PVC) that is based on VolumeSnapshot in a different namespace from the PVC.

@sftim
https://github.com/kubernetes/website/pull/37802/files#diff-f937c4715e7d2fa6db515660503a56c87c317122dc2860873c7022e20e452141R414-R415
I checked the contents of the description again and fixed.

sftim · 2022-11-09T12:16:55Z

BTW:
if this PR is still draft, that's OK - please mark the PR as draft by, eg, editing its title. If it's not draft, please write a PR description.

ttakahashi21 · 2022-11-14T20:11:04Z

/assign msau42

msau42 · 2022-11-15T19:25:33Z

content/en/docs/concepts/storage/persistent-volumes.md


+## Provision of PersistentVolumeClaims from VolumeSnasphot in other namespaces
+
+{{< feature-state for_k8s_version="v1.26" state="alpha" >}}


I think it would be good to incorporate this section into the "Data source references" above.

I see. Fixed ae77a57

xing-yang · 2022-11-21T17:29:41Z

content/en/docs/concepts/storage/persistent-volumes.md

+     resources:
+       requests:
+         storage: 1Gi
+     dataSourceRef2:


dataSourceRef2 -> dataSourceRef

I don't see the fix. Did you reupload?

Sorry, I mistake. I fixed now.

msau42 · 2022-11-21T23:46:39Z

content/en/docs/concepts/storage/persistent-volumes.md

+
+* While the `dataSource` field only allows two specific types of objects, the `dataSourceRef` field allows any non-core object, as well as PersistentVolumeClaim objects.
+* While the `dataSource` field ignores disallowed values (dropping them), the `dataSourceRef` field preserves all values, and generates an error if a disallowed value is specified.
+* While the `dataSource`field only allows local objects, the `dataSourceRef` field allows objects in any namespaces.


I think the first 2 are repeats of L979-L983.

Probably can reword this to say something like

When the CrossNamespaceVolumeDataSource feature is enabled, there are additional differences: * The `dataSource`field only allows local objects, while the `dataSourceRef` field allows objects in any namespaces. * `dataSource` and `dataSourceRef` are not synced.

Thank you for suggestion. Fixed 7cfe0a3

msau42 · 2022-11-21T23:47:45Z

content/en/docs/concepts/storage/persistent-volumes.md

 same namespace, except for core objects other than PVCs. For clusters that have the feature
 gate enabled, use of the `dataSourceRef` is preferred over `dataSource`.

+## Cross Namespace Volume Data Source and data sources


Reword to "Cross namespace data sources"

msau42 · 2022-11-21T23:48:12Z

content/en/docs/concepts/storage/persistent-volumes.md

+     resources:
+       requests:
+         storage: 1Gi
+     dataSourceRef2:


I don't see the fix. Did you reupload?

msau42 · 2022-11-21T23:49:08Z

content/en/docs/concepts/storage/persistent-volumes.md

 the process.

+### Using Cross Namespace Volume Data Source
+


Can you add the feature gate tag here, and also add an example of the ReferenceGrant?

Add feature gate tag and example.

msau42 · 2022-11-22T01:37:54Z

content/en/docs/concepts/storage/persistent-volumes.md

 * The `dataSourceRef` field may contain different types of objects, while the `dataSource` field
  only allows PVCs and VolumeSnapshots.

+When the CrossNamespaceVolumeDataSource feature is enabled, there are additional differences:


Add the backtick, ie CrossNamespaceVolumeDataSource

msau42 · 2022-11-22T01:38:57Z

content/en/docs/concepts/storage/persistent-volumes.md

+When the CrossNamespaceVolumeDataSource feature is enabled, there are additional differences:
+
+* The `dataSource` field only allows local objects, while the `dataSourceRef` field allows objects in any namespaces.  
+* `dataSource` and `dataSourceRef` are not synced.


Oh one more clarification: "When namespace is specified, dataSource and dataSourceRef are not synced"

msau42 · 2022-11-22T01:41:23Z

content/en/docs/concepts/storage/persistent-volumes.md

+   apiVersion: gateway.networking.k8s.io/v1beta1
+   kind: ReferenceGrant
+   metadata:
+     name: bar


a more descriptive name may be "allow-ns1-pvc"

msau42 · 2022-11-22T01:42:37Z

tech review lgtm, just some minor nits. Looks like the PR needs to be rebased as well.

msau42 · 2022-11-23T19:06:56Z

content/en/docs/concepts/storage/persistent-volumes.md

+When the `CrossNamespaceVolumeDataSource` feature is enabled, there are additional differences:
+
+* The `dataSource` field only allows local objects, while the `dataSourceRef` field allows objects in any namespaces.  
+* When namespace is specified, dataSource and dataSourceRef are not synced.


Add the backticks for dataSource and dataSourceRef

leonardpahlke · 2022-11-28T10:36:53Z

cc @krol3

sftim

For beta, we must make sure mention ReferenceGrants in the authorization documentation - see https://kubernetes.io/docs/reference/access-authn-authz/

A mention in https://kubernetes.io/docs/concepts/security/ would be nice-to-have for beta, too.

It'd be even better to make those changes early, for alpha, but it's not mandatory.

sftim · 2022-11-28T12:18:56Z

content/en/docs/concepts/storage/persistent-volumes.md

+## Cross namespace data sources
+{{< feature-state for_k8s_version="v1.26" state="alpha" >}}
+
+Kubernetes supports cross namespace volume data source.


(nit)

Suggested change

Kubernetes supports cross namespace volume data source.

Kubernetes supports cross namespace volume data sources.

sftim · 2022-11-28T12:19:19Z

content/en/docs/concepts/storage/persistent-volumes.md

+To use cross namespace volume datasource, you must enable the `AnyVolumeDataSource` and `CrossNamespaceVolumeDataSource`
+[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) for


(nit)

Suggested change

To use cross namespace volume datasource, you must enable the `AnyVolumeDataSource` and `CrossNamespaceVolumeDataSource`

[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) for

To use cross namespace volume data sources, you must enable the `AnyVolumeDataSource` and `CrossNamespaceVolumeDataSource`

[feature gates](/docs/reference/command-line-tools-reference/feature-gates/) for

sftim · 2022-11-28T12:19:36Z

content/en/docs/concepts/storage/persistent-volumes.md

+To use cross namespace volume datasource, you must enable the `AnyVolumeDataSource` and `CrossNamespaceVolumeDataSource`
+[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) for
+the kube-apiserver, kube-controller-manager.
+Also, you must enable the `CrossNamespaceVolumeDataSource` feature gate for csi-provisioner.


(nit)

Suggested change

Also, you must enable the `CrossNamespaceVolumeDataSource` feature gate for csi-provisioner.

Also, you must enable the `CrossNamespaceVolumeDataSource` feature gate for the csi-provisioner.

sftim · 2022-11-28T12:23:37Z

content/en/docs/concepts/storage/persistent-volumes.md

+Also, you must enable the `CrossNamespaceVolumeDataSource` feature gate for csi-provisioner.
+
+Enabling the `CrossNamespaceVolumeDataSource` feature gate allow you to specify a namespace in the dataSourceRef field.
+{{< note >}}  When a namespace is specified, a gateway.networking.k8s.io/ReferenceGrant object is required in the referent namespace to allow that namespace's owner to accept the reference. See the [ReferenceGrant documentation](https://gateway-api.sigs.k8s.io/api-types/referencegrant/) for details.


Suggested change

{{< note >}} When a namespace is specified, a gateway.networking.k8s.io/ReferenceGrant object is required in the referent namespace to allow that namespace's owner to accept the reference. See the [ReferenceGrant documentation](https://gateway-api.sigs.k8s.io/api-types/referencegrant/) for details.

{{< note >}}

When you specify a namespace for a volume data source, Kubernetes checks for a

ReferenceGrant in the other namespace before accepting the reference.

ReferenceGrant is part of the `gateway.networking.k8s.io` extension APIs.

See [ReferenceGrant](https://gateway-api.sigs.k8s.io/api-types/referencegrant/) in the Gateway API documentation for details.

This means that you must extend your Kubernetes cluster with at least ReferenceGrant from the

Gateway API before you can use this mechanism.

{{< /note >}}

Ideally, we'd also add a glossary entry for ReferenceGrant. We'll definitely need that if this feature moves to beta
and ReferenceGrant is still part of Gateway rather than a general Kubernetes extension.

sftim · 2022-11-28T12:24:32Z

content/en/docs/concepts/storage/persistent-volumes.md

 responsibility of that populator controller to report Events that relate to volume creation and issues during
 the process.

+### Using Cross Namespace Volume Data Source


Suggested change

### Using Cross Namespace Volume Data Source

### Using a cross-namespace volume data source

Even better: add a separate task page, and hyperlink to that task page here. The task can include deploying the CRD for ReferenceGrant.

sftim · 2022-11-28T12:26:33Z

content/en/docs/concepts/storage/persistent-volumes.md

+### Using Cross Namespace Volume Data Source
+{{< feature-state for_k8s_version="v1.26" state="alpha" >}}
+
+Create a gateway.networking.k8s.io/ReferenceGrant to allow the namespace owner to accept the reference.


Suggested change

Create a gateway.networking.k8s.io/ReferenceGrant to allow the namespace owner to accept the reference.

Create a ReferenceGrant to allow the namespace owner to accept the reference.

The API group is gateway.networking.k8s.io/v1beta1. I don't want to mention gateway.networking.k8s.io as a bare thing for an odd reason: we can't name what this is. I mean, it's a domain name but we don't have a name for “an API group, but just the bit before the slash, no version details”.

sftim · 2022-11-28T12:26:53Z

content/en/docs/concepts/storage/persistent-volumes.md

+     namespace: default
+   spec:
+     from:
+     - group: ""


Optionally, add a comment about why this is blank.

Because PersistentVolumeClaim API is core API

(don't tell me - tell the readers!)

sftim · 2022-11-28T12:28:01Z

content/en/docs/reference/command-line-tools-reference/feature-gates.md

  This feature gate will never graduate to stable.
 - `CPUManagerPolicyOptions`: Allow fine-tuning of CPUManager policies.
+- `CrossNamespaceVolumeDataSource`: Enable the usage of cross namespace volume data source
+   to allow you to specify a namespace in the dataSourceRef field.


Suggested change

to allow you to specify a namespace in the dataSourceRef field.

to allow you to specify a source namespace in the `dataSourceRef` field of a

PersistentVolumeClaim.

sftim · 2022-11-28T12:29:16Z

content/en/docs/concepts/storage/persistent-volumes.md

+{{< feature-state for_k8s_version="v1.26" state="alpha" >}}
+
+Create a gateway.networking.k8s.io/ReferenceGrant to allow the namespace owner to accept the reference.
+Users create a populated volume by referring cross namespace volume data source using the `dataSourceRef` field:


Suggested change

Users create a populated volume by referring cross namespace volume data source using the `dataSourceRef` field:

You define a populated volume by specifying a cross namespace volume data source using the `dataSourceRef` field. You must already have a valid ReferenceGrant in the source namespace:

krol3 · 2022-11-28T12:39:57Z

Hi @ttakahashi21, This PR needs a doc review by Mon Nov 28th to get this into the release. Please reach out to required SIGs to get their review. Thank you!
cc: @mickeyboxell

ttakahashi21 · 2022-11-28T13:01:24Z

@sftim I fixed those that needed fixing.

msau42 · 2022-11-28T18:19:59Z

/lgtm

k8s-ci-robot · 2022-11-28T18:20:06Z

LGTM label has been added.

Git tree hash: 1a00b607ebc0d37c6473d91365cd6ce3a22d5c67

sftim

Thanks

/approve

sftim · 2022-11-28T18:29:34Z

content/en/docs/concepts/storage/persistent-volumes.md

+the kube-apiserver, kube-controller-manager.
+Also, you must enable the `CrossNamespaceVolumeDataSource` feature gate for the csi-provisioner.
+
+Enabling the `CrossNamespaceVolumeDataSource` feature gate allow you to specify a namespace in the dataSourceRef field.


(nit)

Suggested change

Enabling the `CrossNamespaceVolumeDataSource` feature gate allow you to specify a namespace in the dataSourceRef field.

Enabling the `CrossNamespaceVolumeDataSource` feature gate allow you to specify a

namespace in the `dataSourceRef` field of a PersistentVolumeClaim.

k8s-ci-robot · 2022-11-28T18:30:05Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sftim

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~content/en/OWNERS~~ [sftim]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot added this to the 1.26 milestone Nov 9, 2022

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Nov 9, 2022

k8s-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Nov 9, 2022

k8s-ci-robot requested a review from kbhawkey November 9, 2022 02:31

k8s-ci-robot added the language/en Issues or PRs related to English language label Nov 9, 2022

k8s-ci-robot requested a review from natalisucks November 9, 2022 02:31

k8s-ci-robot added the sig/docs Categorizes an issue or PR as relevant to SIG Docs. label Nov 9, 2022

ttakahashi21 force-pushed the dev-1.26-KEP-3294-doc branch from d5e3277 to 9eb5a89 Compare November 9, 2022 02:52

mkimuram mentioned this pull request Nov 9, 2022

Provision volumes from cross-namespace snapshots kubernetes/enhancements#3294

Closed

9 tasks

sftim reviewed Nov 9, 2022

View reviewed changes

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 10, 2022

ttakahashi21 changed the title ~~Add document for Provision volumes from cross namespace snapshot~~ [WIP]Add document for Provision volumes from cross namespace snapshot Nov 11, 2022

k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Nov 11, 2022

ttakahashi21 force-pushed the dev-1.26-KEP-3294-doc branch from fc83165 to ac84088 Compare November 14, 2022 19:50

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 14, 2022

ttakahashi21 changed the title ~~[WIP]Add document for Provision volumes from cross namespace snapshot~~ Add document for Provision volumes from cross namespace snapshot Nov 14, 2022

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 14, 2022

k8s-ci-robot assigned msau42 Nov 14, 2022

msau42 reviewed Nov 15, 2022

View reviewed changes

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 17, 2022

xing-yang reviewed Nov 21, 2022

View reviewed changes

msau42 reviewed Nov 21, 2022

View reviewed changes

ttakahashi21 force-pushed the dev-1.26-KEP-3294-doc branch from 3a3578c to 7cfe0a3 Compare November 22, 2022 00:18

msau42 reviewed Nov 22, 2022

View reviewed changes

ttakahashi21 force-pushed the dev-1.26-KEP-3294-doc branch from 7cfe0a3 to b955024 Compare November 23, 2022 01:21

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 23, 2022

msau42 reviewed Nov 23, 2022

View reviewed changes

ttakahashi21 force-pushed the dev-1.26-KEP-3294-doc branch 3 times, most recently from 6b17ed0 to c97489e Compare November 27, 2022 06:42

sftim reviewed Nov 28, 2022

View reviewed changes

Add document for Cross namespace data sources

e837f0f

ttakahashi21 force-pushed the dev-1.26-KEP-3294-doc branch from c97489e to e837f0f Compare November 28, 2022 12:57

ttakahashi21 changed the title ~~Add document for Provision volumes from cross namespace snapshot~~ Add document for Provision volumes from cross namespace data sources Nov 28, 2022

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 28, 2022

sftim reviewed Nov 28, 2022

View reviewed changes

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 28, 2022

k8s-ci-robot merged commit 3d4d447 into kubernetes:dev-1.26 Nov 28, 2022

ttakahashi21 mentioned this pull request Feb 23, 2023

REQUEST: New membership for ttakahashi21 kubernetes/org#4040

Closed

9 tasks

	- `CrossNamespaceVolumeDataSource`: Enable usage of Provision of PVCs from VolumeSnasphot in other namespaces.
	- `CrossNamespaceVolumeDataSource`: Enable provisioning a {{< glossary_tooltip text="PersistentVolumeClaim" term_id="persistent-volume-claim" >}}
	(PVC) that is based on VolumeSnapshot in a different namespace from the PVC.


		## Provision of PersistentVolumeClaims from VolumeSnasphot in other namespaces

		{{< feature-state for_k8s_version="v1.26" state="alpha" >}}

	Kubernetes supports cross namespace volume data source.
	Kubernetes supports cross namespace volume data sources.

		To use cross namespace volume datasource, you must enable the `AnyVolumeDataSource` and `CrossNamespaceVolumeDataSource`
		[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) for

	Also, you must enable the `CrossNamespaceVolumeDataSource` feature gate for csi-provisioner.
	Also, you must enable the `CrossNamespaceVolumeDataSource` feature gate for the csi-provisioner.

-{{< note >}}  When a namespace is specified, a gateway.networking.k8s.io/ReferenceGrant object is required in the referent namespace to allow that namespace's owner to accept the reference. See the [ReferenceGrant documentation](https://gateway-api.sigs.k8s.io/api-types/referencegrant/) for details.
+{{< note >}}
+When you specify a namespace for a volume data source, Kubernetes checks for a
+ReferenceGrant in the other namespace before accepting the reference.
+ReferenceGrant is part of the `gateway.networking.k8s.io` extension APIs.
+See [ReferenceGrant](https://gateway-api.sigs.k8s.io/api-types/referencegrant/) in the Gateway API documentation for details.
+This means that you must extend your Kubernetes cluster with at least ReferenceGrant from the
+Gateway API before you can use this mechanism.
+{{< /note >}}

	### Using Cross Namespace Volume Data Source
	### Using a cross-namespace volume data source

	Create a gateway.networking.k8s.io/ReferenceGrant to allow the namespace owner to accept the reference.
	Create a ReferenceGrant to allow the namespace owner to accept the reference.

	to allow you to specify a namespace in the dataSourceRef field.
	to allow you to specify a source namespace in the `dataSourceRef` field of a
	PersistentVolumeClaim.

	Users create a populated volume by referring cross namespace volume data source using the `dataSourceRef` field:
	You define a populated volume by specifying a cross namespace volume data source using the `dataSourceRef` field. You must already have a valid ReferenceGrant in the source namespace:

	Enabling the `CrossNamespaceVolumeDataSource` feature gate allow you to specify a namespace in the dataSourceRef field.
	Enabling the `CrossNamespaceVolumeDataSource` feature gate allow you to specify a
	namespace in the `dataSourceRef` field of a PersistentVolumeClaim.

Add document for Provision volumes from cross namespace data sources #37802

Add document for Provision volumes from cross namespace data sources #37802

Conversation

ttakahashi21 commented Nov 9, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

netlify bot commented Nov 9, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

👷 Deploy Preview for kubernetes-io-vnext-staging processing.

Uh oh!

sftim left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sftim commented Nov 9, 2022

Uh oh!

ttakahashi21 commented Nov 14, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

msau42 commented Nov 22, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

leonardpahlke commented Nov 28, 2022

Uh oh!

sftim left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sftim Nov 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sftim Nov 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ttakahashi21 commented Nov 9, 2022 •

edited

Loading

netlify bot commented Nov 9, 2022 •

edited

Loading

sftim Nov 28, 2022 •

edited

Loading

sftim Nov 28, 2022 •

edited

Loading