Clean Test Output from User-Facing Messages

[josecelano]

Depends on: #102

📋 Overview

When running cargo test, user-facing output messages (emojis, progress indicators, error messages with tips) appear in stderr, making test output noisy and difficult to read. This issue addresses the need to keep test output clean by default while maintaining observability for debugging.

⚠️ BLOCKED: DO NOT IMPLEMENT YET

This issue depends on a UserOutput refactoring issue (#102) that needs to be implemented first.

🎯 Problem Statement

Current Behavior

Running cargo test produces mixed output in stderr containing:

1. Cargo compilation/test progress (normal, should remain)
2. User-facing application output (problematic, should be silenced in tests)
3. Test names and results (normal, goes to stdout)

Evidence

From captured stderr during cargo test, we see user-facing messages that should not appear:

⏳ First message
✅ Second message
❌ Third message
⏳ Loading configuration from '/tmp/.tmpQ4cTH1/config.json'...
⏳ Creating environment 'test-create-env'...
✅ Configuration template generated: /tmp/.tmpelqHWs/test.json

Next steps:
1. Edit the template file and replace placeholder values...

Root Causes

The issue has two distinct sources:

1. Direct UserOutput Usage in Tests

Tests directly call command handlers that create UserOutput with real stderr. Solution: Use VerbosityLevel::Silent in test contexts.

2. Subprocess Execution in E2E Tests

E2E tests spawn the application as a subprocess, which writes to its own stderr independently. Possible Solutions: Add --silent CLI flag or redirect subprocess stderr to file.

🎯 Goals

• Test output is clean by default (no user-facing messages in stderr)
• User-facing output still available when needed for debugging
• Production behavior unchanged (normal verbose output)
• E2E test output can be captured/silenced when needed
• Solution maintains observability principles

📐 Implementation Strategy

This issue will be addressed incrementally, one message at a time. Each message source will be:

1. Investigated - Find where the message originates
2. Analyzed - Determine the best solution for that specific case
3. Fixed - Apply the appropriate solution
4. Verified - Confirm the message no longer appears in test output

The solution approaches are guidelines, not prescriptions. Each case may require a different approach or combination.

Progress Tracking

• Progress messages (⏳) from direct test usage
• Success messages (✅) from direct test usage
• Error messages (❌) from direct test usage
• Template generation output from subprocess tests
• Environment creation output from subprocess tests
• Destroy command output from subprocess tests

📐 Solution Approaches (Guidelines)

Approach 1: Silent Mode for Direct Test Usage

Add VerbosityLevel::Silent variant and update test contexts to use it by default.

Pros: Simple, maintains existing test structure, no production code changes
Cons: Doesn't solve subprocess output issue

Approach 2: CLI Flag for Subprocess Tests

Add --silent or --quiet flag to suppress all user output.

Pros: Works for subprocess execution, aligns with CLI conventions
Cons: Adds new CLI surface area

Approach 3: Hybrid Solution

Combine both approaches - use Silent mode for direct tests and CLI flag for E2E tests.

Pros: Solves both cases, clean separation of concerns
Cons: More implementation work

📋 Implementation Plan

Strategy: Iterative, Message-by-Message

Each message will follow this workflow:

1. Locate Source - Grep for message text, trace to origin
2. Analyze Context - Understand why/how message is produced
3. Choose Solution - Pick most appropriate approach for this case
4. Implement Fix - Apply chosen solution with tests
5. Verify - Capture stderr, confirm message removed

Phase 1: First Target (Initial Investigation)

Target: "⏳ First message" from direct test usage (simplest case)

Workflow:

• Find exact location producing this output
• Analyze test context and dependencies
• Choose solution (likely Approach 1 or variant)
• Implement fix with tests
• Verify fix with captured stderr

Time estimate: 2-4 hours (includes investigation and design)

Subsequent Targets: Iterative Progression

After Phase 1, continue with remaining message categories using the same workflow.

✅ Acceptance Criteria

Functional Requirements

• Running cargo test produces clean stderr (no user-facing messages)
• Cargo's own progress messages remain visible
• Tests can suppress user-facing output without breaking functionality
• CLI provides mechanism to control output verbosity (if needed for E2E tests)
• Error messages still appear when commands fail
• E2E test failures show subprocess stderr for debugging

Quality Requirements

• All existing tests pass
• Tests added to verify clean test output behavior
• Pre-commit checks pass: ./scripts/pre-commit.sh
• Documentation updated as needed for any new features or patterns
• No regression in production output behavior

User Experience

• Test output is easy to read (no noise)
• Failed tests show sufficient context for debugging
• CLI help documentation updated if new flags or features added
• Error messages remain clear and actionable in all output modes

📚 Related Documentation

• Development Principles - User Friendliness and Observability
• Error Handling Guide - Error message requirements
• Testing Conventions - Test output patterns

⏱️ Time Estimate

Phase 1 (First Target): 2-4 hours

Total Project: Unknown at this stage. Will be estimated after Phase 1 completion.

Note: This iterative approach allows for better time estimation as each message case is investigated and understood.