Torrust Tracker Deploy

This Rust application provides automated deployment infrastructure for Torrust tracker projects. It manages VM provisioning and cloud-init execution using LXD containers, with the goal of finding the best solution for creating VMs that support cloud-init both locally (development) and in CI environments (GitHub Actions).

🎯 Project Goals

• ✅ Create VMs supporting cloud-init locally and in GitHub runners
• ✅ Test cloud-init execution and verification
• ✅ Support Docker Compose inside VMs (planned)
• ✅ Fast, easy to install and use solutions
• ❌ No nested virtualization dependency (CI compatibility)

🔧 Available Approaches

This repository uses LXD containers for virtualization:

☁️ LXD Containers (templates/tofu/lxd/) - OFFICIAL

• Technology: System containers with cloud-init support
• Status: ✅ Official provider - Guaranteed GitHub Actions compatibility
• Best for: CI/CD environments, fast provisioning, local development
• Requirements: No special virtualization needed

📖 See detailed documentation →

� Provider Comparison

📖 See detailed comparison →

Feature	LXD (Official)	Multipass (Experimental)
GitHub Actions Support	✅ Guaranteed	⚠️ Undocumented
Nested Virtualization	❌ Not needed	✅ Required
Boot Time	✅ ~5-10s	❌ ~30-60s
Resource Usage	✅ Lower	❌ Higher
Isolation Level	🔶 Process-level	✅ Hardware-level

🚀 Quick Start

Prerequisites

This is a Rust application that automates deployment infrastructure using OpenTofu and Ansible.

Install the required tools:

# Check installations
lxd version && tofu version && ansible --version && cargo --version

Missing tools? See detailed installation guides:

• 📖 OpenTofu Setup Guide →
• 📖 Ansible Setup Guide →

Quick install:

# Install Rust (if not already installed)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Install LXD
sudo snap install lxd && sudo lxd init --auto && sudo usermod -a -G lxd $USER && newgrp lxd

# Install OpenTofu
curl -fsSL https://get.opentofu.org/install-opentofu.sh | sudo bash

# Install Ansible
sudo apt install ansible

Usage

Main Application

The main application provides usage instructions:

# Build and run the application
cargo run

# Or install and run directly
cargo install --path .
torrust-tracker-deploy

Development Tasks

This project includes convenient scripts for common development tasks:

# Run all linters (markdown, YAML, TOML, shell scripts, Rust)
cargo run --bin linter all

Or run individual linters:

cargo run --bin linter markdown    # Markdown linting
cargo run --bin linter yaml        # YAML linting
cargo run --bin linter toml        # TOML linting
cargo run --bin linter clippy      # Rust code analysis
cargo run --bin linter rustfmt     # Rust formatting check
cargo run --bin linter shellcheck  # Shell script linting

📖 See linting documentation →

Running E2E Tests

Use the E2E tests binary to run automated infrastructure tests:

# Run the comprehensive E2E tests
cargo run --bin e2e-tests-full

# Keep the test environment after completion
cargo run --bin e2e-tests-full -- --keep

# Use custom templates directory
cargo run --bin e2e-tests-full -- --templates-dir ./custom/templates

# See all available options
cargo run --bin e2e-tests-full -- --help

Manual Deployment Steps

If you prefer manual deployment instead of using the E2E tests:

1. Deploy Infrastructure

# Navigate to LXD configuration
cd templates/tofu/lxd

# Initialize and deploy
tofu init && tofu apply

2. Configure with Ansible

# Navigate to Ansible configuration
cd ../../ansible

# Update inventory.yml with the VM's IP from step 1
# Then run the verification playbook
ansible-playbook wait-cloud-init.yml

# Install Docker on the VM
ansible-playbook install-docker.yml

# Install Docker Compose on the VM (optional)
ansible-playbook install-docker-compose.yml

3. Verify Deployment

lxc list torrust-tracker-vm

# Access the container directly
lxc exec torrust-tracker-vm -- /bin/bash

# Test SSH connection
ssh -i ~/.ssh/testing_rsa torrust@<VM_IP>

# Verify Docker installation
lxc exec torrust-vm -- docker --version
lxc exec torrust-vm -- docker run --rm hello-world

# Verify Docker Compose installation (if installed)
lxc exec torrust-vm -- docker-compose --version

🎭 Infrastructure Workflow

1. Provision: OpenTofu creates and configures VMs with cloud-init
2. Configure: Ansible connects to VMs and executes management tasks
3. Verify: Automated checks ensure proper setup and functionality

Phase	Tool	Purpose
Infrastructure	OpenTofu	VM provisioning and cloud-init setup
Configuration	Ansible	Task execution and configuration management
Verification	Ansible Playbooks	System checks and validation

📖 See detailed Ansible documentation →

🧪 Testing in GitHub Actions

The repository includes comprehensive GitHub Actions workflows for CI testing:

• .github/workflows/test-e2e.yml - End-to-End Tests - Runs automated E2E tests using the Rust binary
• .github/workflows/test-lxd-provision.yml - Tests LXD container provisioning

📊 Current Status

✅ Completed

• LXD container provisioning (local + GitHub Actions)
• Cloud-init support for LXD containers
• OpenTofu infrastructure as code
• Ansible configuration management setup
• Basic cloud-init verification playbook
• Docker installation playbook
• Docker Compose installation playbook
• Automated testing workflows
• End-to-End (E2E) deployment infrastructure and workflows

🔄 In Progress

• Extended Ansible playbooks for application deployment
• Performance benchmarking
• Official GitHub Actions nested virtualization clarification

📋 Planned

• Additional VM providers evaluation
• Integration with Torrust application testing
• Multi-architecture support (ARM64)

📁 Repository Structure

├── .github/                  # CI/CD workflows and GitHub configuration
│   └── workflows/           # GitHub Actions workflow files
├── .vscode/                 # VS Code workspace configuration
├── build/                   # 📁 Generated runtime configs (git-ignored)
│   ├── tofu/                # 🏗️ Runtime OpenTofu configs
│   └── ansible/             # 🤖 Runtime Ansible configs
├── data/                    # Data files and templates
│   └── templates/           # Template sources for generation
│       ├── ansible/         # Ansible template sources
│       └── tofu/            # OpenTofu template sources
├── docs/                    # 📖 Detailed documentation
│   ├── tech-stack/          # Technology-specific documentation
│   │   ├── opentofu.md      # OpenTofu installation and usage
│   │   ├── ansible.md       # Ansible installation and usage
│   │   └── lxd.md          # LXD system containers
│   ├── decisions/           # Architecture Decision Records (ADRs)
│   ├── contributing/        # Contributing guidelines and conventions
│   │   ├── README.md        # Main contributing guide
│   │   ├── branching.md     # Git branching conventions
│   │   ├── commit-process.md # Commit process and pre-commit checks
│   │   └── testing.md       # Testing conventions
│   ├── research/            # Research and analysis documents
│   └── *.md                 # Various documentation files
├── examples/                # Example configurations and usage
├── fixtures/                # Test fixtures and sample data
│   ├── testing_rsa*         # SSH key pair for testing
│   └── tofu/               # OpenTofu test fixtures
├── packages/                # Rust workspace packages
│   └── linting/            # Linting utilities package
│       └── src/            # Linting implementation source code
├── scripts/                 # Development and utility scripts
│   └── setup/              # Installation scripts for dependencies
├── src/                     # 🦀 Main Rust application source code
│   ├── main.rs             # Main application binary entry point
│   ├── lib.rs              # Library root module
│   ├── bin/                # Binary executables
│   │   ├── linter.rs       # Unified linting command interface
│   │   └── e2e_tests.rs    # End-to-end testing binary
│   ├── ansible/            # Ansible integration modules
│   ├── command_wrappers/   # External command wrapper modules
│   ├── commands/           # CLI command implementations
│   ├── config/             # Configuration handling
│   ├── e2e/                # End-to-end testing infrastructure
│   ├── remote_actions/     # Remote system management actions
│   ├── steps/              # Deployment step implementations
│   ├── template/           # Template processing and rendering
│   └── tofu/               # OpenTofu integration modules
├── templates/               # 📁 Template configurations (git-tracked)
│   ├── tofu/               # 🏗️ OpenTofu/Terraform templates
│   │   └── lxd/            # LXD container template configuration
│   └── ansible/            # 🤖 Ansible playbook templates
├── tests/                  # Integration and system tests
├── target/                 # 🦀 Rust build artifacts (git-ignored)
├── Cargo.toml             # Rust workspace configuration
├── Cargo.lock             # Rust dependency lock file
├── cspell.json            # Spell checking configuration
├── project-words.txt      # Custom dictionary for spell checking
├── .markdownlint.json     # Markdown linting configuration
├── .taplo.toml            # TOML formatting configuration
├── .yamllint-ci.yml       # YAML linting configuration for CI
├── README.md              # This file - project overview
├── LICENSE                # Project license
└── .gitignore             # Git ignore rules

📚 Documentation

• 🤝 Contributing Guide - Git workflow, commit process, and linting conventions
• 📖 Documentation Organization Guide - How documentation is organized and where to contribute
• 📋 Structured Logging Implementation Plan - Implementation plan for hierarchical logging with tracing spans
• 📖 OpenTofu Setup Guide - Installation, common commands, and best practices
• 📖 Ansible Setup Guide - Installation, configuration, and project usage
• 📖 VM Providers Comparison - Detailed comparison and decision rationale

🔮 Next Steps

This is a basic setup. Future enhancements could include:

• Multiple VMs for different testing scenarios
• Custom images with pre-installed Torrust components
• Network configuration for multi-VM setups
• Enhanced CI/CD integration with nested virtualization
• Automated testing scripts for Torrust applications