Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 88 additions & 0 deletions content/rent-gpu/deploy-container/with-cli.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -281,6 +281,94 @@ Here is the logs for the Lease ID: 44
...
```

### SSH Access to Deployment

To access your deployment via SSH for more convenient management and advanced operations, follow these steps:

1. **Generate SSH Key Pair (if you don't already have one)**

Generate an SSH key pair on your local machine:

```sh
ssh-keygen -t rsa -b 4096 -C "[email protected]"
```

Follow the prompts to save the key. By default, the keys will be saved in `~/.ssh/id_rsa` (private key) and `~/.ssh/id_rsa.pub` (public key).

2. **Create a Deployment with SSH Support**

Ensure your deployment uses an SSH-compatible image and exposes port 22. Here's an example configuration:

```yaml
services:
pytorch-server:
image: spheronfdn/pytorch:latest
expose:
- port: 22
as: 22
to:
- global: true
```

3. **Set Your SSH Public Key**

Make sure to wait for the deployment to be ready before setting the SSH key. Add your public SSH key to be used with deployments:

```sh
sphnctl deployment ssh set-key ~/.ssh/id_rsa.pub
```

The result will show:
```
Public key set successfully
```

4. **Enable SSH for Your Deployment**

Enable SSH access for your specific deployment:

```sh
sphnctl deployment ssh enable --lid <LID>
```

For example:
```sh
sphnctl deployment ssh enable --lid 44
```

The output will provide connection details:
```
Enabling SSH access to the deployment...
User: root
Deployment ID: 44
Public Key: /Users/username/.ssh/id_rsa.pub
Waiting for deployment to be ready...
Deployment is ready...
SSH port: 32496
SSH host: provider.thor.gpufarm.xyz
Please use the following command to connect to the deployment:
ssh -i <private-key> -p 32496 [email protected]
```

5. **Connect via SSH**

Use the provided command to connect to your deployment:

```sh
ssh -i ~/.ssh/id_rsa -p 32496 [email protected]
```

<Callout type="info">
**NOTE:** SSH access requires that your deployment use an SSH-compatible image and that the provider supports SSH connections. Some Spheron-supported images with SSH access include:

- [spheronfdn/pytorch:latest](https://github.com/spheronFdn/awesome-spheron/blob/main/pytorch/spheron.yaml)
- [spheronfdn/tensorflow:latest](https://github.com/spheronFdn/awesome-spheron/blob/main/tensorflow/spheron.yaml)
- [spheronfdn/cuda:12.3.1-devel-ubuntu22.04](https://github.com/spheronFdn/awesome-spheron/blob/main/nvidia-cuda-devel/spheron.yaml)
- [spheronfdn/cuda:12.4.1-cudnn-devel-ubuntu22.04](https://github.com/spheronFdn/awesome-spheron/blob/main/nvidia-cudnn-devel/spheron.yaml)
- [spheronfdn/ubuntu:22.04](https://github.com/spheronFdn/awesome-spheron/blob/main/ubuntu22/spheron.yaml)
- [spheronfdn/ubuntu:24.04](https://github.com/spheronFdn/awesome-spheron/blob/main/ubuntu24/spheron.yaml)
</Callout>

### Connect to the deployment shell

To connect to the deployment shell, you need to run this command:
Expand Down
1 change: 0 additions & 1 deletion content/rent-gpu/fizz-vs-provider.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,6 @@ sphnctl deployment logs --lid 123

- Basic logging capabilities
- Startup logs available - to check what's happening during the startup of your application
- No real-time streaming
- Limited filtering options
- Detailed event tracking - you can see all the events happening in your deployment

Expand Down
137 changes: 132 additions & 5 deletions content/rent-gpu/icl.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ It's structured into several key sections:
10. [Shared Memory (SHM) Support](/rent-gpu/icl#10-shared-memory-shm-support)
11. [Advanced Placement Attributes](/rent-gpu/icl#11-advanced-placement-attributes)
12. [Mac Deployment Support](/rent-gpu/icl#12-mac-deployment-support)
13. [GPU VRAM Minimum Requirements](/rent-gpu/icl#13-gpu-vram-minimum-requirements)

<Callout type="info">
**NOTE:** For examples of deployment configurations:
Expand Down Expand Up @@ -74,8 +75,6 @@ When configuring port exposure for your services, keep these points in mind:
as: 80
to:
- global: true
accept:
- www.yoursite.com
```

### Port Range
Expand Down Expand Up @@ -439,6 +438,42 @@ gpu:
</Callout>


#### Configuring VRAM Requirements

You can specify the minimum VRAM requirement using the `req_vram` attribute in your GPU configuration. Here's an example:

```yaml
profiles:
compute:
gpu-intensive:
resources:
cpu:
units: 4.0
memory:
size: 8Gi
storage:
size: 20Gi
gpu:
units: 1
attributes:
vendor:
nvidia:
- model: rtx3090
req_vram: ">=80"
```

The `req_vram` attribute accepts a string value with a comparison operator followed by the required VRAM amount in percentage. The supported comparison operators are:

- `>=` - Greater than or equal to
- `>` - Greater than
- `<=` - Less than or equal to
- `<` - Less than

Examples:
- `req_vram: ">=80"` - Requires at least 80% of VRAM
- `req_vram: ">50"` - Requires more than 50% of VRAM
- `req_vram: "<=20"` - Requires 20% of VRAM or less
- `req_vram: "<50"` - Requires less than 50% of VRAM

## 7. Private Container Registry Integration

Expand Down Expand Up @@ -629,9 +664,17 @@ attributes:
region: us-east
```

You can also specify multiple regions using a semicolon-separated list:

```yaml
attributes:
region: us-east;ap-south;eu-west
```

This is particularly useful when you need to:
- Minimize latency for users in specific geographic areas
- Ensure your application runs in a specific region
- Deploy your workload across multiple regions for redundancy

<Callout type="info">
The region must match one of Spheron's [supported region codes](/rent-gpu/supports#supported-regions). If not specified, your application can be deployed in any available region.
Expand All @@ -645,10 +688,18 @@ attributes:
exclude_region: us-east
```

You can also exclude multiple regions using a semicolon-separated list:

```yaml
attributes:
exclude_region: us-east;ap-south;eu-west
```

This is useful when you:
- Want to exclude a specific region from deployment
- Need to ensure your application does not run in a specific region
- Are testing region-specific functionality
- Have compliance requirements that prohibit deployment in certain regions

#### 3. Provider Selection
The `desired_provider` attribute lets you specify a particular provider using their blockchain address:
Expand All @@ -658,13 +709,21 @@ attributes:
desired_provider: "0x1234...5678" # Replace with actual provider address
```

You can also specify multiple providers as a semicolon-separated list:

```yaml
attributes:
desired_provider: "0x1234...5678;0xabcd...1234;0xefgh...5678" # Multiple provider addresses
```

This is helpful when you:
- Have had good experiences with a specific provider
- Want to maintain consistency across deployments
- Need specific hardware or capabilities that a provider offers
- Want to distribute your workload across multiple preferred providers

<Callout type="info">
This works in both `provider` and `fizz` modes. In `fizz` mode, it will select fizz nodes that are connected to your chosen provider.
This works in both `provider` and `fizz` modes. In `fizz` mode, it will select fizz nodes that are connected to your chosen provider(s).
</Callout>

#### 3. Fizz Node Selection
Expand All @@ -675,10 +734,18 @@ attributes:
desired_fizz: "0xabcd...ef12" # Replace with actual fizz node address
```

Similar to provider selection, you can specify multiple fizz nodes as a semicolon-separated list:

```yaml
attributes:
desired_fizz: "0xabcd...ef12;0x5678...90ab;0xcdef...3456" # Multiple fizz node addresses
```

This is useful when you:
- Want to deploy to a specific node you trust
- Need to maintain application state on a particular node
- Want to deploy to specific nodes you trust
- Need to maintain application state on particular nodes
- Are testing node-specific functionality
- Want to distribute your workload across multiple preferred fizz nodes

#### 4. Bandwidth Selection
<Callout type="info">
Expand Down Expand Up @@ -707,6 +774,7 @@ profiles:
region: us-east # Geographic region
desired_provider: "0x1234...5678" # Specific provider
desired_fizz: "0xabcd...ef12" # Specific fizz node
bandwidth: 100mbps # Minimum bandwidth
pricing:
web:
token: uSPON # uSPON is the only supported token for Spheron deployments
Expand Down Expand Up @@ -776,3 +844,62 @@ Mac deployments are particularly beneficial for:
<Callout type="info">
**Note:** The availability of specific Mac models depends on what's currently available in the Spheron network. Please check the supported models before configuring your deployment.
</Callout>

## 13. GPU VRAM Minimum Requirements

<Callout type="warning">
**Important:** This feature is only applicable for fizz mode deployments and not for provider deployments.
</Callout>

When deploying GPU workloads to fizz nodes, you can specify minimum VRAM (Video RAM) requirements to ensure you get nodes with sufficient available memory for your application. This helps maximize your workload efficiency and prevents deployment to nodes with insufficient VRAM.

### Configuring VRAM Requirements

You can specify the minimum VRAM requirement using the `req_vram` attribute in your GPU configuration. Here's an example:

```yaml
profiles:
compute:
gpu-intensive:
resources:
cpu:
units: 4.0
memory:
size: 8Gi
storage:
size: 20Gi
gpu:
units: 1
attributes:
vendor:
nvidia:
- model: rtx3090
req_vram: ">=80"
```

### Using Comparison Operators

The `req_vram` attribute accepts a string value with a comparison operator followed by the required VRAM amount in percentage. The supported comparison operators are:

- `>=` - Greater than or equal to
- `>` - Greater than
- `<=` - Less than or equal to
- `<` - Less than

Examples:
- `req_vram: ">=80"` - Requires at least 80% of VRAM
- `req_vram: ">50"` - Requires more than 50% of VRAM
- `req_vram: "<=20"` - Requires 20% of VRAM or less
- `req_vram: "<50"` - Requires less than 50% of VRAM

### Benefits

Specifying VRAM requirements provides several advantages:

1. **Resource Optimization**: Ensures your application gets the GPU memory it needs to run efficiently.
2. **Deployment Reliability**: Prevents your workload from running on underpowered nodes that might crash or perform poorly.
3. **Cost Efficiency**: Helps you match your workload to appropriate resources without overprovisioning.

<Callout type="info">
**Note:** When using the `req_vram` attribute, make sure the value you specify is between 0% and 100% of the GPU's total VRAM. For example, don't request more than 100% of the VRAM or less than 0%.
</Callout>
Loading