diff --git a/content/docs/ref/runner.md b/content/docs/ref/runner.md index 3cf810a5..cf5a8647 100644 --- a/content/docs/ref/runner.md +++ b/content/docs/ref/runner.md @@ -75,6 +75,52 @@ Any [generic option](/doc/ref) in addition to: ## Examples +### Using `--cloud-permission-set` + +> Currently this feature is only implemented when using GCP or AWS. + +You can predefine a set of permissions via an [AWS role]() or a +[GCP service account]() that is assosiated with the created `cml runner` +instance. + +This can enable credentialless access to your `s3` or `gs` dvc remote. +Additionally, you can use this access other resources from that cloud provider +like AWS' Elastic Container Registry or GCP's Artifact Registry, so that you can +push and pull custom docker images. + +Other examples, using AWS, could include accessing data in: + +- Secrets Manager +- DynamoDB +- Redshfit + +#### Example "Permission Sets" + + + +stuff + +```json +{ + "stuff": "here" +} +``` + + + + +stuff + +- list of roles + + + + +> Caveat for `--cloud-permission-set` on GCP: using this feature will likely +> require and additional role be added to your `cml runner` credentials +> `roles/ServiceAccountUser` or ensure the invoker has the permission +> `iam.serviceAccount.actAs` on the targeted Service Account. + ### Using `--cloud-ssh-private` 1. Generate a new RSA PEM private key for debugging purposes: diff --git a/content/docs/self-hosted-runners.md b/content/docs/self-hosted-runners.md index 5fe6e088..921f6cef 100644 --- a/content/docs/self-hosted-runners.md +++ b/content/docs/self-hosted-runners.md @@ -127,6 +127,47 @@ below for details on the `secrets` required. `cml send-comment` from a job, the only requirement is to [have CML installed](/doc/install). +## Accessing DVC remotes on your CML cloud runner + +If you're using an Object Storage remotes like `s3` or `gs` from AWS/GCP it's +easy to allow DVC programatic access without the use of dedicated credentials. + +Besides reducing overhead in managing additional keys, you can save in network +costs, and have options to increase transfer speeds. For example, looking at +AWS, we can get [free network transfers](https://aws.amazon.com/s3/pricing/) +from `s3` to `ec2` within the same region. + +These `cml runner` commands fit right in with the above examples. For a more +detailed breakdown checkout [the advanced guide](/cool/link). + + + + +```bash +cml runner \ + --cloud=aws \ + --cloud-region=us-west \ + --cloud-type=p2.xlarge \ + --cloud-permission-set=arn:aws:iam::1234567890:instance-profile/dvc-s3-access \ + --labels=cml-gpu +``` + + + + + +```bash +cml runner \ + --cloud=gcp \ + --cloud-region=us-west \ + --cloud-type=someinstance+gpu? \ + --cloud-permission-set=dvc-sa@myproject.iam.gserviceaccount.com,scopes=storage-rw \ + --labels=cml-gpu +``` + + + + ## Docker Images The CML Docker images (`docker://iterativeai/cml` or