Skip to content

Commit a2246fc

Browse files
authored
feat: add persistent storage support in ICL documentation
Merge pull request #13 from spheronFdn/rekpero/persistent-storage
2 parents 6bb9076 + a0e7a51 commit a2246fc

File tree

1 file changed

+108
-0
lines changed

1 file changed

+108
-0
lines changed

content/rent-gpu/icl.mdx

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -956,3 +956,111 @@ Specifying VRAM requirements provides several advantages:
956956
<Callout type="info">
957957
**Note:** When using the `req_vram` attribute, make sure the value you specify is between 0% and 100% of the GPU's total VRAM. For example, don't request more than 100% of the VRAM or less than 0%.
958958
</Callout>
959+
960+
## 14. Persistent Storage
961+
962+
Spheron's Infrastructure Composition Language (ICL) supports persistent storage, allowing you to maintain data across container restarts and server failures during your lease duration. This feature is essential for applications that need to preserve data, such as databases, file systems, or any application that generates important data that should survive container restarts.
963+
964+
### How Persistent Storage Works
965+
966+
Persistent storage in Spheron provides the following behavior:
967+
968+
1. **During Lease Duration**: Files stored in the persistent mount point will survive container restarts, server failures, or application crashes. If your application exits due to Out of Memory (OOM) errors or other issues, the persistent data remains intact.
969+
970+
2. **Lease Closure**: When your lease expires or is manually closed, all persistent storage data is permanently destroyed and cannot be recovered.
971+
972+
3. **Mount Point Persistence**: Any files written to the specified mount point will be persisted to the underlying storage system.
973+
974+
### Configuring Persistent Storage in Services
975+
976+
To enable persistent storage in your service, add the `params.storage` section to your service definition:
977+
978+
```yaml
979+
services:
980+
myapp:
981+
image: myregistry.com/myapp:latest
982+
params:
983+
storage:
984+
default:
985+
mount: /home/jovyan/work
986+
readOnly: false
987+
expose:
988+
- port: 8080
989+
as: 80
990+
to:
991+
- global: true
992+
```
993+
994+
#### Service Storage Parameters
995+
996+
| Parameter | Required | Description |
997+
|------------|----------|-------------|
998+
| `mount` | Yes | The mount point where persistent storage will be available inside the container |
999+
| `readOnly` | No | Whether the storage should be mounted as read-only. Defaults to `false` |
1000+
1001+
### Configuring Persistent Storage in Compute Profiles
1002+
1003+
In your compute profile, you need to define the persistent storage configuration:
1004+
1005+
```yaml
1006+
profiles:
1007+
compute:
1008+
myapp:
1009+
resources:
1010+
cpu:
1011+
units: 2
1012+
memory:
1013+
size: 4Gi
1014+
storage:
1015+
name: "default"
1016+
size: 200Gi
1017+
attributes:
1018+
persistent: true
1019+
class: beta3
1020+
```
1021+
1022+
#### Storage Configuration Parameters
1023+
1024+
| Parameter | Required | Description |
1025+
|--------------|----------|-------------|
1026+
| `name` | Yes | Must be set to "default" to match the service storage reference |
1027+
| `size` | Yes | The amount of persistent storage to allocate (e.g., 200Gi) |
1028+
| `persistent` | Yes | Must be set to `true` to enable persistent storage |
1029+
| `class` | Yes | Must be set to "beta3" (currently the only supported storage class) |
1030+
1031+
### Complete Persistent Storage Example
1032+
1033+
Here's a complete example of a deployment with persistent storage - [persistent storage example](https://github.com/spheronFdn/awesome-spheron/blob/main/examples/spheron.persistent.yaml)
1034+
1035+
### Important Notes
1036+
1037+
<Callout type="warning">
1038+
**Storage Lifecycle**: Persistent storage is tied to your lease duration. When the lease ends, all persistent data is permanently deleted and cannot be recovered.
1039+
</Callout>
1040+
1041+
<Callout type="info">
1042+
**Current Limitations**:
1043+
- The storage class must be set to "beta3" as it's the only currently supported class
1044+
- The storage name must be "default" to properly reference it in the service configuration
1045+
- Persistent storage is only available for provider (or secure) deployments
1046+
</Callout>
1047+
1048+
### Use Cases
1049+
1050+
Persistent storage is ideal for:
1051+
1052+
1. **Development Environments**: Jupyter notebooks, VS Code workspaces, or development tools that need to preserve project files
1053+
2. **Data Processing**: Applications that process large datasets and need to store intermediate results
1054+
3. **Databases**: Running databases that need to persist data between container restarts
1055+
4. **File Servers**: Applications that serve files and need to maintain file integrity
1056+
5. **AI/ML Training**: Machine learning workflows that need to save model checkpoints and training data
1057+
1058+
### Best Practices
1059+
1060+
1. **Backup Important Data**: Since persistent storage is destroyed when the lease ends, regularly backup critical data to external storage
1061+
2. **Monitor Storage Usage**: Keep track of your storage usage to avoid running out of space
1062+
3. **Use Appropriate Mount Points**: Choose mount points that align with your application's data directory structure
1063+
4. **Consider Storage Size**: Allocate sufficient storage space for your application's needs, keeping in mind the cost implications
1064+
1065+
For a complete working example, refer to the [persistent storage example](https://github.com/spheronFdn/awesome-spheron/blob/main/examples/spheron.persistent.yaml) in the Spheron examples repository.
1066+

0 commit comments

Comments
 (0)