Proposal: Migrate Automation from Makefile/sh to a Test-Driven Meson/Perl Toolchain

[da2ce7]

The Problem

The project's current automation, based on Makefile and POSIX shell scripts, has been effective but has several inherent limitations that conflict with the Twelve-Factor principles we are adopting for the application itself:

1. Implicit Dependencies: The toolchain relies on system tools (make, awk, grep) whose presence and version are not explicitly declared or verified, leading to potentially non-deterministic CI/CD and contributor environments.
2. Brittle Logic: Shell scripts frequently rely on parsing the text output of other programs, creating fragile contracts that can break between OS or tool versions.
3. No Clear Build/Run Separation: The make apply target, for example, mixes infrastructure provisioning, configuration, and execution without a distinct, verifiable "build" step for the automation logic itself.

This issue proposes a strategic migration to address these shortcomings and elevate our automation tooling to a first-class, professionally engineered component of the project.

The Proposal

The associated Pull Request [#16] introduces a series of Architecture Decision Records (ADRs) that outline a comprehensive migration plan. The core of the proposal is to:

1. Adopt a New Toolchain (ADR-004):

   • Interface: Use the Meson Build System as the single, declarative entry point for all project tasks.
   • Logic: Migrate imperative shell script logic to a structured Modern Perl codebase.
   • Stability: Benchmark all tools and dependencies against the versions shipped in the current Debian Stable release (including official backports).

2. Implement a Test-Driven Structure (ADR-005):

   • Create a new top-level automation/ directory to house all imperative code and its tests.
   • Mandate a test-driven standard, with meson test serving as the validation command for the automation logic itself.
   • Execute an atomic "flag day" migration, completely removing the legacy Makefile system in a single changeset to provide immediate clarity.

3. Enforce Quality and Security Standards (ADR-006, ADR-007):

   • Establish strict coding standards for Perl (use strict;, use warnings;, use autodie;).
   • Harden all executable scripts by default with Perl's taint mode (-T).
   • Integrate static analysis (Perl::Critic) and unit testing (prove) into the meson test quality gate.
   • Define clear principles for modular, robust, and readable Meson build files.

Alternatives Considered (Contra-Analysis)

The choice of Meson and Perl over more conventional alternatives was deliberate and based on the core principles outlined in the ADRs.

1. Why Not Python?

Python is the modern default for automation and scripting.

• Pros: Widely known, excellent libraries, strong community support.
• Reason for Rejection: The Python ecosystem strongly favors language-specific package managers (pip, Poetry). This creates a direct conflict with our proposed "System-Package-Only" policy, which is designed to ensure a stable, curated supply chain for our infrastructure tooling. Adopting Python would necessitate managing virtual environments (venv) and dependencies from PyPI, adding a layer of complexity and moving away from the stability of the system's native package manager (apt install python3-boto3 vs. pip install boto3).

2. Why Not Stick with Enhanced Shell Scripts?

We could keep the existing system but enforce stricter standards (e.g., shellcheck).

• Pros: Ubiquitous, low barrier to entry, no new dependencies for contributors.
• Reason for Rejection: This fails to solve the core problems. Shell is fundamentally poor at handling structured data (requiring more external dependencies like jq), has brittle error handling, and relies on fragile text-parsing to interact with other tools. To achieve the robustness we need, we would essentially be re-implementing a "real" programming language poorly within shell.

3. Why Not Go or Rust?

These compiled languages offer performance and type safety.

• Pros: Single static binaries, excellent performance, strong type systems (especially Rust).
• Reason for Rejection: The primary role of our automation is to be "glue code"—orchestrating external tools like tofu, docker, and virsh. The development cycle for compiled languages is significantly slower for this kind of iterative scripting. The added complexity of their build systems and type systems (e.g., Rust's borrow checker) provides diminishing returns for our specific use case, which does not require high-performance computation.

Conclusion: Why Meson and Perl are the Right Fit

• Perl: It remains a best-in-class "glue" language. Its strengths in text processing and system command orchestration are ideal for our needs. Crucially, its mature ecosystem of system administration modules (like Sys::Virt) is widely packaged and maintained by Debian, directly supporting our System-Package-Only policy. Its security-focused taint mode (-T) is a unique advantage for this domain.
• Meson: It is more than a task runner; it is a true build system. Its first-class support for dependency discovery (find_program, dependency) and project configuration (meson setup) provides the explicit, verifiable setup stage that the current Makefile system lacks.

Key Benefits

• Reliability: Explicit, verifiable dependencies (meson setup) and a test suite for the automation code will lead to fewer bugs and a more robust system.
• Maintainability: A well-structured, consistent, and statically analyzed codebase is easier to read, review, and extend.
• Security: Taint mode provides a powerful, language-level defense against a class of command-injection vulnerabilities.

Acknowledged Trade-offs

The primary trade-off is sacrificing the "zero-setup" convenience of a pre-installed make. Contributors will now be required to install the Meson/Perl toolchain via their system package manager as a first step. This is a conscious decision to prioritize explicit, verifiable correctness over initial convenience.

Call to Action

We are seeking review and feedback on the proposed ADRs in pull request [#16]. The discussion should focus on whether this strategic direction is sound for the project's long-term health. If this proposal is accepted, it will serve as the foundation for the subsequent implementation work.