torrust · josecelano · Jul 31, 2025 · Jul 31, 2025 · Jul 31, 2025 · Jul 31, 2025
diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md
@@ -182,6 +182,20 @@ torrust-tracker-demo/
 - **Cross-cutting**: Project-wide documentation and ADRs
 - **Architecture Decisions**: Documented design choices and rationale
 
+#### Architecture Decision Records (ADRs)
+
+Key design decisions are documented in `docs/adr/`. Contributors should review relevant ADRs when working on related features:
+
+| ADR                                                                                 | Title                                    | Description                                                           |
+| ----------------------------------------------------------------------------------- | ---------------------------------------- | --------------------------------------------------------------------- |
+| [ADR-001](../docs/adr/001-makefile-location.md)                                     | Makefile Location                        | Why the main Makefile is at repository root                           |
+| [ADR-002](../docs/adr/002-docker-for-all-services.md)                               | Docker for All Services                  | Why we use Docker for all services including UDP tracker              |
+| [ADR-003](../docs/adr/003-use-mysql-over-mariadb.md)                                | Use MySQL Over MariaDB                   | Database backend selection rationale                                  |
+| [ADR-004](../docs/adr/004-configuration-approach-files-vs-environment-variables.md) | Configuration Approach                   | Files vs environment variables for configuration                      |
+| [ADR-005](../docs/adr/005-sudo-cache-management-for-infrastructure-operations.md)   | Sudo Cache Management                    | Managing sudo credentials during infrastructure operations            |
+| [ADR-006](../docs/adr/006-ssl-certificate-generation-strategy.md)                   | SSL Certificate Generation Strategy      | Approach for SSL certificate management                               |
+| [ADR-007](../docs/adr/007-two-level-environment-variable-structure.md)              | Two-Level Environment Variable Structure | Security-focused separation of infrastructure and container variables |
+
 ## 🛠️ Development Workflow
 
 ### Quick Start for Contributors
@@ -284,9 +298,11 @@ The twelve-factor **Build, Release, Run** stages apply to the application deploy
 
 - **Release Stage**: Combine built application with environment-specific configuration
 
-  - Apply environment variables and configuration files
+  - Apply environment variables and configuration files using **two-level structure** (see [ADR-007](../docs/adr/007-two-level-environment-variable-structure.md))
+    - **Level 1**: Main environment files (`infrastructure/config/environments/`) for all deployment processes
+    - **Level 2**: Docker environment files (generated from templates) for container runtime only
   - Combine application artifacts with runtime configuration
-  - Prepare deployment-ready releases
+  - Prepare deployment-ready releases with security-focused variable filtering
 
 - **Run Stage**: Execute the application in the runtime environment
   - Start application processes (tracker binary, background jobs)

diff --git a/README.md b/README.md
@@ -90,6 +90,51 @@ clear separation between infrastructure provisioning and application deployment:
 - **Environment-based Configuration**: Template system with `local.env.tpl` and `production.env.tpl`
 - **Build/Release/Run Stages**: Proper separation of configuration processing, deployment, and execution
 
+## 🚀 Deployment Process Overview
+
+This project follows a **four-step deployment process** based on twelve-factor app principles:
+
+### 1. **Configure Environment Variables**
+
+Customize deployment settings using environment-specific templates:
+
+- `infrastructure/config/environments/local.env.tpl` → `local.env`
+- `infrastructure/config/environments/production.env.tpl` → `production.env`
+- Contains ALL deployment configuration (infrastructure, SSL, backups, application secrets)
+
+### 2. **Provision Infrastructure**
+
+Create and configure the target environment:
+
+- **VM Creation**: Deploy virtual machine with specified resources
+- **System Setup**: cloud-init installs dependencies, configures firewall, sets up users
+- **Platform Preparation**: Ready environment for application deployment
+
+### 3. **Deploy Application** (Build + Release + Run Stages)
+
+Deploy the application stack following twelve-factor principles:
+
+- **Build**: Prepare application artifacts and configuration files
+- **Release**: Generate SSL certificates, create Docker environment files, copy configs
+- **Run**: Start Docker Compose stack (tracker, MySQL, Nginx, Grafana, Prometheus)
+
+### 4. **Validation**
+
+Verify deployment health and functionality:
+
+- Service health checks (HTTP/UDP endpoints)
+- Database connectivity validation
+- Monitoring dashboard accessibility
+- End-to-end smoke testing
+
+### Quick Commands
+
+```bash
+make infra-apply ENVIRONMENT=local      # Steps 1-2: Configure + Provision
+make app-deploy ENVIRONMENT=local       # Step 3: Deploy Application
+make app-health-check ENVIRONMENT=local # Step 4: Validation
+```
+
 ## Demo Tracker
 
 - **HTTP Tracker**: <https://tracker.torrust-demo.com/announce>

diff --git a/docs/adr/007-two-level-environment-variable-structure.md b/docs/adr/007-two-level-environment-variable-structure.md
@@ -0,0 +1,232 @@
+# ADR-007: Two-Level Environment Variable Structure
+
+## Status
+
+Accepted
+
+## Context
+
+As part of implementing twelve-factor app methodology, we need a clear approach for
+managing environment variables throughout the deployment process. The system has
+evolved to use a two-level environment variable structure that serves different
+purposes and security requirements.
+
+Currently, the project uses environment variables at two distinct levels:
+
+1. **Main Environment Variables**: Used for the entire deployment process
+2. **Docker Compose Environment Variables**: Used only for running containers
+
+This separation has emerged organically but lacks clear documentation, leading to
+potential confusion about where variables should be defined and how they flow
+through the system.
+
+## Decision
+
+We will formalize and document a **two-level environment variable structure**
+with clear separation of concerns:
+
+### Level 1: Main Environment Variables
+
+**Purpose**: Complete deployment configuration  
+**Location**: `infrastructure/config/environments/`  
+**Examples**: `local.env`, `production.env`  
+**Scope**: All deployment processes
+
+**Contents**:
+
+- Infrastructure configuration (VM specs, network settings)
+- SSL certificate configuration (domains, Let's Encrypt email)
+- Database credentials and connection parameters
+- Application API tokens and secrets
+- Backup and monitoring configuration
+- Build and deployment automation settings
+
+**Usage**:
+
+- Sourced by deployment scripts (`provision-infrastructure.sh`, `deploy-app.sh`)
+- Used for template rendering (cloud-init, configuration files)
+- Contains variables for infrastructure operations (SSL generation, backups)
+- Includes variables that containers never need to see
+
+### Level 2: Docker Compose Environment Variables
+
+**Purpose**: Container runtime configuration  
+**Template**: `infrastructure/config/templates/docker-compose.env.tpl`  
+**Generated File**: `.env` (in application directory)  
+**Scope**: Docker Compose and running containers only
+
+**Contents** (filtered subset from Level 1):
+
+- Database connection strings for application containers
+- Application API tokens needed by running services
+- Docker runtime configuration (USER_ID)
+- Service-specific configuration (Grafana admin credentials)
+- Container environment overrides
+
+**Filtering Criteria**:
+
+- **Include**: Variables directly used by containerized applications
+- **Exclude**: Infrastructure-only variables (SSL domains, backup settings)
+- **Exclude**: Build-time variables not needed at runtime
+- **Security**: Minimize attack surface by only exposing necessary variables
+
+## Template Transformation Process
+
+```text
+Level 1: Main Environment Variables
+├── infrastructure/config/environments/local.env.tpl
+├── infrastructure/config/environments/production.env.tpl
+└── (user creates) local.env or production.env
+                    │
+                    ▼ (template processing)
+Level 2: Docker Environment Variables
+├── infrastructure/config/templates/docker-compose.env.tpl
+└── (generated) application/.env
+```
+
+**Processing Flow**:
+
+1. User creates environment file from template (e.g., `local.env`)
+2. Deployment script sources the main environment file
+3. Template processor generates `docker-compose.env` from template
+4. Docker Compose uses the generated `.env` file for container variables
+
+## Rationale
+
+### Security Benefits
+
+- **Principle of Least Privilege**: Containers only receive variables they need
+- **Reduced Attack Surface**: Infrastructure secrets not exposed to application containers
+- **Separation of Concerns**: Infrastructure and application secrets handled differently
+
+### Operational Benefits
+
+- **Clear Responsibility**: Infrastructure variables vs. application variables
+- **Easier Debugging**: Know where to look for specific types of configuration
+- **Template Flexibility**: Can generate different container environments from same base config
+- **Deployment Isolation**: Infrastructure operations don't leak sensitive data to containers
+
+### Examples
+
+**Level 1 Only (Infrastructure Variables)**:
+
+```bash
+# SSL configuration (not needed in containers)
+SSL_DOMAIN="tracker.example.com"
+SSL_EMAIL="[email protected]"
+ENABLE_SSL_AUTOMATION="true"
+
+# Backup configuration (not needed in containers)
+ENABLE_DB_BACKUPS="true"
+BACKUP_RETENTION_DAYS="30"
+
+# Infrastructure specifications (not needed in containers)
+VM_MEMORY="4096"
+VM_VCPUS="4"
+```
+
+**Level 2 (Container Variables - Filtered from Level 1)**:
+
+```bash
+# Database connection (needed by tracker container)
+MYSQL_ROOT_PASSWORD="secure_root_password"
+MYSQL_PASSWORD="secure_user_password"
+TORRUST_TRACKER_CONFIG_OVERRIDE_CORE__DATABASE__PATH="mysql://torrust:${MYSQL_PASSWORD}@mysql:3306/torrust_tracker"
+
+# API tokens (needed by application)
+TRACKER_ADMIN_TOKEN="secure_api_token"
+
+# Runtime configuration (needed by containers)
+USER_ID="1000"
+```
+
+## Implementation Guidelines
+
+### For Infrastructure Scripts
+
+```bash
+# Source the main environment file
+source "infrastructure/config/environments/${ENVIRONMENT}.env"
+
+# Use all variables for infrastructure operations
+generate_ssl_certificates "$SSL_DOMAIN" "$SSL_EMAIL"
+configure_backups "$ENABLE_DB_BACKUPS" "$BACKUP_RETENTION_DAYS"
+```
+
+### For Template Processing
+
+```bash
+# Generate Docker environment file from template
+envsubst < "infrastructure/config/templates/docker-compose.env.tpl" > "application/.env"
+```
+
+### For Container Configuration
+
+```bash
+# Docker Compose command (in deploy-app.sh)
+docker compose --env-file /var/lib/torrust/compose/.env up -d
+
+# The .env file contains only container-relevant variables
+# and is passed to Docker Compose via the --env-file flag
+```
+
+## Benefits
+
+1. **Security**: Reduced container attack surface
+2. **Clarity**: Clear separation between infrastructure and application concerns
+3. **Maintainability**: Easier to understand what variables are used where
+4. **Flexibility**: Can generate different container environments from same base
+5. **Compliance**: Aligns with twelve-factor configuration principles
+6. **Debugging**: Easier to troubleshoot configuration issues
+
+## Trade-offs
+
+### Accepted Complexity
+
+- **Two Files to Maintain**: Requires keeping template and source in sync
+- **Template Processing**: Additional step in deployment process
+- **Learning Curve**: Contributors must understand the two-level structure
+
+### Mitigated Risks
+
+- **Template Drift**: Validation scripts check template consistency
+- **Missing Variables**: Docker Compose will fail fast if required variables are missing
+- **Documentation**: This ADR and inline comments clarify the structure
+
+## Consequences
+
+### For Contributors
+
+- Must understand which level to modify for different types of changes
+- Infrastructure changes: Edit main environment templates
+- Container configuration: Edit Docker environment template
+- New variables: Consider which level(s) need the variable
+
+### For Deployment
+
+- All deployment scripts use Level 1 (main environment)
+- Docker Compose only sees Level 2 (filtered environment)
+- Template processing is automatic during deployment
+- No manual synchronization required
+
+### For Security
+
+- Infrastructure secrets isolated from application containers
+- Container compromise doesn't expose infrastructure configuration
+- Easier security auditing of container-exposed variables
+
+## Alternative Considered
+
+**Single-Level Environment Variables**: Using one environment file for everything.
+
+**Rejected because**:
+
+- Security: All variables exposed to containers
+- Complexity: Difficult to determine which variables containers actually need
+- Maintenance: Changes to infrastructure configuration could affect containers unnecessarily
+
+## References
+
+- [Twelve-Factor App: Config](https://12factor.net/config)
+- [ADR-004: Configuration Approach](./004-configuration-approach-files-vs-environment-variables.md)
+- [Docker Compose Environment Variables](https://docs.docker.com/compose/environment-variables/)
diff --git a/docs/guides/cloud-deployment-guide.md b/docs/guides/cloud-deployment-guide.md
@@ -12,6 +12,53 @@ and testing. Hetzner Cloud support is planned as the next implementation target.
 The process combines Infrastructure as Code with application deployment automation to
 provide a streamlined deployment experience, following twelve-factor app methodology.
 
+## Deployment Process
+
+This project implements a **four-step deployment workflow** aligned with twelve-factor app principles:
+
+### Step 1: Configure Environment Variables
+
+Create environment-specific configuration from templates:
+
+- **Local Development**: `infrastructure/config/environments/local.env.tpl` → `local.env`
+- **Production**: `infrastructure/config/environments/production.env.tpl` → `production.env`
+
+The environment file contains **all deployment configuration**, including:
+
+- Infrastructure settings (VM specs, network configuration)
+- Application secrets (database passwords, API tokens)
+- SSL certificate configuration (domains, email for Let's Encrypt)
+- Backup and monitoring settings
+
+### Step 2: Provision Infrastructure
+
+Deploy and configure the target environment:
+
+- **VM Creation**: Deploy virtual machine with specified resources
+- **System Dependencies**: cloud-init installs Docker, configures firewall, creates users
+- **Network Setup**: Configure firewall rules, SSH access, system security
+- **Platform Readiness**: Environment prepared for application deployment
+
+### Step 3: Deploy Application (Build + Release + Run)
+
+Deploy the application stack following twelve-factor methodology:
+
+- **Build Stage**: Prepare application artifacts and configuration templates
+- **Release Stage**:
+  - Generate SSL certificates (Let's Encrypt or self-signed)
+  - Create Docker environment files from templates
+  - Copy application configurations to target VM
+- **Run Stage**: Start Docker Compose stack (Torrust Tracker, MySQL, Nginx, Grafana, Prometheus)
+
+### Step 4: Validation
+
+Verify deployment health and functionality:
+
+- **Service Health**: HTTP/UDP endpoint availability checks
+- **Database Connectivity**: MySQL connection and schema validation
+- **Monitoring Access**: Grafana dashboard accessibility
+- **End-to-end Testing**: Tracker announce/scrape functionality
+
 ## Prerequisites
 
 ### Local Requirements