fix: [#28] improve infrastructure provisioning UX and documentation

- Infrastructure waiting logic: Added proper VM IP and cloud-init waiting
- SSH key auto-detection: Documented automatic detection of ~/.ssh/torrust_rsa.pub
- Environment file naming: Clarified flexible naming conventions (not mandatory format)
- Output display fix: Fixed cosmetic issue showing actual VM IP instead of 'No IP assigned yet'
- Documentation updates: Enhanced cloud deployment guide with SSH and environment details

Key improvements:
✅ Infrastructure provisioning now waits for full readiness by default
✅ Clear SSH key auto-detection documentation and comments
✅ Flexible environment file naming (my-dev.env, local-test.env, etc.)
✅ Fixed final output to display correct VM IP address (192.168.122.21)
✅ Enhanced user experience with automatic waiting and progress indicators

Files changed:
- infrastructure/scripts/provision-infrastructure.sh: Added waiting logic and fixed IP display
- infrastructure/config/templates/environments/: Updated SSH key documentation
- docs/guides/cloud-deployment-guide.md: Comprehensive SSH and environment documentation
- infrastructure/config/environments/README.md: Environment file naming clarification

Author: josecelano

docs/guides/cloud-deployment-guide.md

@@ -20,8 +20,9 @@ This project implements a **four-step deployment workflow** aligned with twelve-
 
 Create environment-specific configuration from templates:
 
-- **Local Development**: `infrastructure/config/environments/local.env.tpl` → `local.env`
-- **Production**: `infrastructure/config/environments/production.env.tpl` → `production.env`
+- **Development Environment**: `development-libvirt.env` (default) or custom name like `my-dev.env`
+- **End-to-end Testing**: `e2e-libvirt.env` (default) or custom name like `ci-test.env`
+- **Production Environment**: `production-hetzner.env` (default) or custom name like `prod.env`
 
 The environment file contains **all deployment configuration**, including:
 
@@ -66,8 +67,37 @@ Verify deployment health and functionality:
 - **OpenTofu** (or Terraform) installed
 - **Git** for repository access
 - **SSH client** for server access
+- **SSH key pair** for VM access (see SSH Key Configuration below)
 - **Domain name** (required for HTTPS certificates in production)
 
+#### SSH Key Configuration
+
+The deployment system requires an SSH public key for secure VM access. The system
+automatically detects SSH keys from these locations (in order):
+
+1. `~/.ssh/torrust_rsa.pub` (recommended - dedicated key for Torrust deployments)
+2. `~/.ssh/id_rsa.pub` (common default SSH key)
+3. `~/.ssh/id_ed25519.pub` (Ed25519 SSH key)
+4. `~/.ssh/id_ecdsa.pub` (ECDSA SSH key)
+
+**Recommended Setup**:
+
+```bash
+# Generate dedicated SSH key for Torrust deployments
+ssh-keygen -t rsa -b 4096 -f ~/.ssh/torrust_rsa -C "[email protected]"
+
+# The public key (~/.ssh/torrust_rsa.pub) will be auto-detected
+# The private key (~/.ssh/torrust_rsa) will be used for SSH connections
+```
+
+**Alternative Options**:
+
+- **Use existing key**: Copy your existing public key to `~/.ssh/torrust_rsa.pub`
+- **Manual configuration**: Set `SSH_PUBLIC_KEY` in your environment file
+- **Environment variable**: Export the key content as an environment variable
+
+If no SSH key is found, the deployment will provide detailed error messages with setup instructions.
+
 ### Cloud Provider Requirements (For Future Implementation)
 
 When cloud providers are implemented, they will need:
@@ -90,10 +120,18 @@ cloud-agnostic to facilitate adding cloud providers that support cloud-init in t
 
 ## Quick Start
 
-### Current Implementation: Local Development
+### Current Implementation: Development Environment (KVM/libvirt)
 
-The current implementation supports local KVM/libvirt deployment, which is perfect
-for development, testing, and understanding the system before cloud deployment.
+The current implementation supports local KVM/libvirt deployment using the **development**
+environment type, which is perfect for development, testing, and understanding the system
+before cloud deployment.
+
+**Environment Types vs Environment Files**:
+
+- **Environment Types**: `development`, `testing`, `e2e`, `staging`, `production`
+- **Environment Files**: Any name you choose (e.g., `my-dev.env`, `local-test.env`)
+- **Default Format**: `{environment-type}-{provider}.env` (when using generation scripts)
+- **Examples**: `development-libvirt.env`, `production-hetzner.env`, `my-custom-setup.env`
 
 ### 1. Clone and Setup
 
@@ -105,22 +143,29 @@ cd torrust-tracker-demo
 # Install dependencies (Ubuntu/Debian)
 make install-deps
 
-# Configure SSH access for VMs
-make infra-config-local
+# Setup SSH key for VM access (if you don't have ~/.ssh/torrust_rsa.pub)
+ssh-keygen -t rsa -b 4096 -f ~/.ssh/torrust_rsa -C "[email protected]"
+
+# The system will auto-detect ~/.ssh/torrust_rsa.pub during deployment
 ```
 
 ### 2. Local Testing with KVM/libvirt
 
 ```bash
-# Test deployment locally with KVM
+# Test deployment locally with KVM using development environment (default naming)
 # Commands wait for full readiness by default
-make infra-apply ENVIRONMENT=local
-make app-deploy ENVIRONMENT=local
+make infra-apply ENVIRONMENT_TYPE=development ENVIRONMENT_FILE=development-libvirt
+make app-deploy ENVIRONMENT_TYPE=development ENVIRONMENT_FILE=development-libvirt
+make app-health-check
+
+# Alternative: Use custom file names
+make infra-apply ENVIRONMENT_TYPE=development ENVIRONMENT_FILE=my-custom-dev
+make app-deploy ENVIRONMENT_TYPE=development ENVIRONMENT_FILE=my-custom-dev
 make app-health-check
 
 # Advanced users: Skip waiting for faster execution
-make infra-apply ENVIRONMENT=local SKIP_WAIT=true
-make app-deploy ENVIRONMENT=local SKIP_WAIT=true
+make infra-apply ENVIRONMENT_TYPE=development ENVIRONMENT_FILE=development-libvirt SKIP_WAIT=true
+make app-deploy ENVIRONMENT_TYPE=development ENVIRONMENT_FILE=development-libvirt SKIP_WAIT=true
 make app-health-check
 
 # Access the local instance via SSH
@@ -134,17 +179,20 @@ make vm-ssh
 make infra-destroy
 ```
 
+**Note**: `ENVIRONMENT_FILE` can be any filename (without `.env` extension).
+The system looks for the file in `infrastructure/config/environments/{filename}.env`.
+
 ### 3. Cloud Deployment (Planned - Hetzner)
 
 **Note**: Cloud deployment is not yet implemented. The following commands show the
 planned interface for future Hetzner Cloud deployment:
 
 ```bash
-# Planned: Deploy infrastructure to Hetzner Cloud
-make infra-apply ENVIRONMENT=production PROVIDER=hetzner
+# Planned: Deploy infrastructure to Hetzner Cloud using production environment
+make infra-apply ENVIRONMENT_TYPE=production ENVIRONMENT_FILE=production-hetzner
 
 # Planned: Deploy application services
-make app-deploy ENVIRONMENT=production
+make app-deploy ENVIRONMENT_TYPE=production ENVIRONMENT_FILE=production-hetzner
 
 # Validate deployment
 make app-health-check
@@ -315,9 +363,14 @@ default, providing a much better user experience:
 
 ```bash
 # Each command waits for full readiness by default
-make infra-apply ENVIRONMENT=local    # Waits for VM IP + cloud-init completion
-make app-deploy ENVIRONMENT=local     # Waits for all services to be healthy
-make app-health-check                 # Validates everything is working
+make infra-apply ENVIRONMENT_TYPE=development ENVIRONMENT_FILE=development-libvirt
+make app-deploy ENVIRONMENT_TYPE=development ENVIRONMENT_FILE=development-libvirt
+make app-health-check
+
+# What each command does:
+# infra-apply: Waits for VM IP + cloud-init completion
+# app-deploy:  Waits for all services to be healthy
+# health-check: Validates everything is working
 ```
 
 **Key improvements**:
@@ -331,17 +384,17 @@ make app-health-check                 # Validates everything is working
 
 ```bash
 # Skip waiting for faster execution (original behavior)
-make infra-apply ENVIRONMENT=local SKIP_WAIT=true
-make app-deploy ENVIRONMENT=local SKIP_WAIT=true
+make infra-apply ENVIRONMENT_TYPE=development ENVIRONMENT_FILE=development-libvirt SKIP_WAIT=true
+make app-deploy ENVIRONMENT_TYPE=development ENVIRONMENT_FILE=development-libvirt SKIP_WAIT=true
 ```
 
 ### Infrastructure Deployment
 
 The infrastructure deployment creates and configures the VM:
 
 ```bash
-# Deploy infrastructure
-make infra-apply ENVIRONMENT=production
+# Deploy infrastructure using development environment
+make infra-apply ENVIRONMENT_TYPE=development ENVIRONMENT_FILE=development-libvirt
 
 # What this does:
 # 1. Creates VM with Ubuntu 24.04
@@ -359,8 +412,8 @@ make infra-apply ENVIRONMENT=production
 The application deployment sets up all services:
 
 ```bash
-# Deploy application
-make app-deploy ENVIRONMENT=production
+# Deploy application using development environment
+make app-deploy ENVIRONMENT_TYPE=development ENVIRONMENT_FILE=development-libvirt
 
 # What this does:
 # 1. Clones torrust-tracker-demo repository
@@ -478,18 +531,18 @@ docker compose exec -T mysql mysql -u root -p torrust_tracker
 
 ## Environment Configuration
 
-### Local Development
+### Development Environment (Local Testing)
 
-For local testing and development:
+For local testing and development using KVM/libvirt:
 
 ```bash
-# Use local environment
-make infra-apply ENVIRONMENT=local
-make app-deploy ENVIRONMENT=local
+# Use development environment type with libvirt provider
+make infra-apply ENVIRONMENT_TYPE=development ENVIRONMENT_FILE=development-libvirt
+make app-deploy ENVIRONMENT_TYPE=development ENVIRONMENT_FILE=development-libvirt
 
 # Features enabled:
 # - HTTPS with self-signed certificates (automatic)
-# - Local domain names (tracker.test.local, grafana.test.local)
+# - Local domain names (test.local)
 # - Full monitoring with Grafana and Prometheus
 # - MySQL database (same as production)
 # - All production features except trusted SSL certificates
@@ -535,10 +588,10 @@ Generate the production configuration template:
 
 ```bash
 # Generate production configuration template with placeholders
-make infra-config-production
+make infra-config-production PROVIDER=hetzner
 ```
 
-This will create `infrastructure/config/environments/production.env` with secure placeholder
+This will create `infrastructure/config/environments/production-hetzner.env` with secure placeholder
 values that need to be replaced with your actual configuration.
 
 #### Step 3: Replace Placeholder Values
@@ -547,7 +600,7 @@ Edit the generated production environment file with your secure secrets and doma
 
 ```bash
 # Edit the production configuration
-vim infrastructure/config/environments/production.env
+vim infrastructure/config/environments/production-hetzner.env
 ```
 
 **Replace these placeholder values with your actual configuration**:
@@ -594,9 +647,9 @@ make infra-config-production
 planned interface for future production deployments:
 
 ```bash
-# Planned: Use production environment
-make infra-apply ENVIRONMENT=production DOMAIN=your-domain.com
-make app-deploy ENVIRONMENT=production
+# Planned: Use production environment type with Hetzner provider
+make infra-apply ENVIRONMENT_TYPE=production ENVIRONMENT_FILE=production-hetzner
+make app-deploy ENVIRONMENT_TYPE=production ENVIRONMENT_FILE=production-hetzner
 
 # Planned features:
 # - HTTPS support (with automated certificate setup)

infrastructure/config/environments/README.md

@@ -21,19 +21,27 @@ These files are generated from templates in `../templates/environments/` and con
 
 ## File Naming Convention
 
-Environment files follow the pattern: `{environment}-{provider}.env`
+Environment files can use any naming convention you prefer. The generation scripts use the
+**default pattern**: `{environment-type}-{provider}.env`
 
-Examples:
+**Default Examples** (when using generation scripts):
 
 - `development-libvirt.env` - Development environment using libvirt provider
 - `staging-hetzner.env` - Staging environment using Hetzner Cloud provider
 - `production-hetzner.env` - Production environment using Hetzner Cloud provider
 
+**Custom Examples** (user-defined names):
+
+- `my-dev-setup.env` - Custom development configuration
+- `local-testing.env` - Local testing environment
+- `client-prod.env` - Client-specific production setup
+
 ## Creating Environment Files
 
-1. **Use templates**: Copy from `../templates/environments/{environment}.env.tpl`
-2. **Use generation scripts**: Run `infrastructure/scripts/configure-env.sh {environment}`
-3. **Follow naming convention**: Always include the provider suffix
+1. **Use generation scripts**: Run `infrastructure/scripts/configure-env.sh {environment} {provider}`
+   (creates files with default naming: `{environment}-{provider}.env`)
+2. **Use templates manually**: Copy from `../templates/environments/{environment}.env.tpl`
+3. **Custom naming**: Name your files however you prefer, just ensure they have `.env` extension
 
 ## Security Best Practices
 

infrastructure/config/templates/environments/base.env.tpl

@@ -18,7 +18,12 @@ VM_MEMORY=${VM_MEMORY}
 VM_VCPUS=${VM_VCPUS}
 VM_DISK_SIZE=${VM_DISK_SIZE}
 PERSISTENT_DATA_SIZE=${PERSISTENT_DATA_SIZE}
+
+# SSH Public Key for VM access
+# Leave empty for auto-detection from ~/.ssh/torrust_rsa.pub (recommended)
+# Or set manually: SSH_PUBLIC_KEY="ssh-rsa AAAAB3NzaC1yc2EAAAA... [email protected]"
 SSH_PUBLIC_KEY=${SSH_PUBLIC_KEY}
+
 USE_MINIMAL_CONFIG=${USE_MINIMAL_CONFIG}
 
 # === SECRETS (DOCKER SERVICES) ===

infrastructure/config/templates/environments/development.defaults

@@ -12,7 +12,7 @@ VM_MEMORY="2048"
 VM_VCPUS="2"
 VM_DISK_SIZE="20"
 PERSISTENT_DATA_SIZE="20"
-SSH_PUBLIC_KEY=""  # Will be auto-detected or user must configure
+SSH_PUBLIC_KEY=""  # Leave empty - auto-detected from ~/.ssh/torrust_rsa.pub during deployment
 USE_MINIMAL_CONFIG="false"
 
 TEMPLATE_PROCESSING_VARS="

infrastructure/scripts/provision-infrastructure.sh

@@ -181,17 +181,51 @@ provision_infrastructure() {
     if [[ "${ACTION}" == "apply" ]]; then
         log_success "Infrastructure provisioning completed"
 
-        # Try to get VM IP from Terraform output
+        # Wait for VM readiness if not skipped
+        if [[ "${SKIP_WAIT}" != "true" ]]; then
+            # Wait for VM IP assignment
+            if ! wait_for_vm_ip "${ENVIRONMENT_TYPE}" "${ENVIRONMENT_FILE}" "${PROJECT_ROOT}"; then
+                log_error "Failed to get VM IP - infrastructure may not be fully ready"
+                return 1
+            fi
+            
+            # Wait for cloud-init completion
+            if ! wait_for_cloud_init_completion "${ENVIRONMENT_TYPE}" "${ENVIRONMENT_FILE}" "${PROJECT_ROOT}"; then
+                log_error "Failed to wait for cloud-init completion - VM may not be fully ready"
+                return 1
+            fi
+            
+            log_success "✅ Infrastructure is fully ready"
+        fi
+        
+        # Get VM IP for final display - use the reliable traditional output approach
         local vm_ip
-        vm_ip=$(tofu output -raw vm_ip 2>/dev/null || echo "")
 
-        if [[ -n "${vm_ip}" ]]; then
+        # Change to terraform directory and get IP from tofu output
+        cd "${PROJECT_ROOT}/infrastructure/terraform" || return 1
+        vm_ip=$(tofu output vm_ip 2>/dev/null | tr -d '"' || echo "")
+        
+        # If Terraform doesn't have it, get directly from libvirt
+        if [[ -z "${vm_ip}" || "${vm_ip}" == "No IP assigned yet" ]]; then
+            vm_ip=$(get_vm_ip_from_libvirt "${ENVIRONMENT_TYPE}" "${ENVIRONMENT_FILE}" "${PROJECT_ROOT}")
+        fi
+        
+        if [[ -n "${vm_ip}" && "${vm_ip}" != "No IP assigned yet" ]]; then
             log_success "VM IP Address: ${vm_ip}"
             log_info ""
-            log_info "Next steps:"
-            log_info "1. Wait for cloud-init to complete (may take 2-3 minutes)"
-            log_info "2. Connect via SSH: ssh torrust@${vm_ip}"
-            log_info "3. Deploy application: make app-deploy ENVIRONMENT_TYPE=${ENVIRONMENT_TYPE} ENVIRONMENT_FILE=${ENVIRONMENT_FILE}"
+            if [[ "${SKIP_WAIT}" == "true" ]]; then
+                log_info "Next steps (SKIP_WAIT enabled):"
+                log_info "1. Wait for cloud-init to complete (may take 2-3 minutes)"
+                log_info "2. Connect via SSH: ssh torrust@${vm_ip}"
+                log_info "3. Deploy application: make app-deploy ENVIRONMENT_TYPE=${ENVIRONMENT_TYPE} ENVIRONMENT_FILE=${ENVIRONMENT_FILE}"
+            else
+                log_info "Next steps:"
+                log_info "1. Deploy application: make app-deploy ENVIRONMENT_TYPE=${ENVIRONMENT_TYPE} ENVIRONMENT_FILE=${ENVIRONMENT_FILE}"
+                log_info "2. Access VM: ssh torrust@${vm_ip}"
+            fi
+        else
+            log_warning "Could not determine VM IP address for display"
+            log_info "Check VM status with: make infra-status ENVIRONMENT_TYPE=${ENVIRONMENT_TYPE} ENVIRONMENT_FILE=${ENVIRONMENT_FILE}"
         fi
     fi
 }