Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions content/docs/ref/runner.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,52 @@ Any [generic option](/doc/ref) in addition to:

## Examples

### Using `--cloud-permission-set`

> Currently this feature is only implemented when using GCP or AWS.

You can predefine a set of permissions via an [AWS role]() or a
[GCP service account]() that is assosiated with the created `cml runner`
instance.

This can enable credentialless access to your `s3` or `gs` dvc remote.
Additionally, you can use this access other resources from that cloud provider
like AWS' Elastic Container Registry or GCP's Artifact Registry, so that you can
push and pull custom docker images.

Other examples, using AWS, could include accessing data in:

- Secrets Manager
- DynamoDB
- Redshfit

#### Example "Permission Sets"

<toggle>
<tab title="AWS">
stuff

```json
{
"stuff": "here"
}
```

</tab>
<tab title="GCP">

stuff

- list of roles

</tab>
</toggle>

> Caveat for `--cloud-permission-set` on GCP: using this feature will likely
> require and additional role be added to your `cml runner` credentials
> `roles/ServiceAccountUser` or ensure the invoker has the permission
> `iam.serviceAccount.actAs` on the targeted Service Account.

### Using `--cloud-ssh-private`

1. Generate a new RSA PEM private key for debugging purposes:
Expand Down
41 changes: 41 additions & 0 deletions content/docs/self-hosted-runners.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,47 @@ below for details on the `secrets` required.
`cml send-comment` from a job, the only requirement is to
[have CML installed](/doc/install).

## Accessing DVC remotes on your CML cloud runner

If you're using an Object Storage remotes like `s3` or `gs` from AWS/GCP it's
easy to allow DVC programatic access without the use of dedicated credentials.

Besides reducing overhead in managing additional keys, you can save in network
costs, and have options to increase transfer speeds. For example, looking at
AWS, we can get [free network transfers](https://aws.amazon.com/s3/pricing/)
from `s3` to `ec2` within the same region.

These `cml runner` commands fit right in with the above examples. For a more
detailed breakdown checkout [the advanced guide](/cool/link).

<toggle>
<tab title="AWS">

```bash
cml runner \
--cloud=aws \
--cloud-region=us-west \
--cloud-type=p2.xlarge \
--cloud-permission-set=arn:aws:iam::1234567890:instance-profile/dvc-s3-access \
--labels=cml-gpu
```

</tab>

<tab title="GCP">

```bash
cml runner \
--cloud=gcp \
--cloud-region=us-west \
--cloud-type=someinstance+gpu? \
[email protected],scopes=storage-rw \
--labels=cml-gpu
```

</tab>
</toggle>

## Docker Images

The CML Docker images (`docker://iterativeai/cml` or
Expand Down