Skip to content

Commit 7390cd5

Browse files
committed
docs: update test-case-enhancement.md
1 parent e6c468d commit 7390cd5

File tree

6 files changed

+43
-505
lines changed

6 files changed

+43
-505
lines changed
Lines changed: 43 additions & 178 deletions
Original file line numberDiff line numberDiff line change
@@ -1,213 +1,78 @@
11
# Test Case Enhancement Rules
22

3-
## Assistant Workflow for Adding Comprehensive Test Cases
3+
## Simple Enhancement Workflow
44

5-
### Automated Workflow (Recommended)
5+
When user requests test case enhancement:
66

7-
For single problem enhancement:
7+
### 1. Problem Resolution
88

9-
1. **Use automation script**: `./.amazonq/rules/test-case-enhancement/scripts/enhance_single_problem.sh <problem_name>`
10-
2. **Verify results**: `./.amazonq/rules/test-case-enhancement/scripts/verify_enhancement.sh <problem_name>`
9+
- Use active file context or user-provided problem name
10+
- If unclear, run: `poetry run python .templates/check_test_cases.py --threshold=10 --max=1`
1111

12-
For batch enhancement:
13-
14-
1. **Find problems**: `./.amazonq/rules/test-case-enhancement/scripts/find_problems_needing_enhancement.sh`
15-
2. **Batch enhance**: `./.amazonq/rules/test-case-enhancement/scripts/enhance_batch_problems.sh`
16-
17-
### Manual Workflow (Fallback)
18-
19-
When user requests to enhance test cases for a problem, the assistant will:
20-
21-
### 1. Problem Resolution (Priority Order)
22-
23-
- **FIRST**: Try to resolve from context - check active file path or user-provided problem name
24-
- **SECOND**: If context resolution fails, THEN run `poetry run python .templates/check_test_cases.py --threshold=10 --max=1` to auto-detect 1 problem with <10 test cases
25-
- **LAST**: If both above fail, ask user to explicitly specify problem name
26-
27-
### 2. Test Case Generation
28-
29-
- Read `leetcode/{problem_name}/README.md` for problem understanding
30-
- Analyze existing test cases in `leetcode/{problem_name}/tests.py`
31-
- Generate comprehensive test cases covering:
32-
- **Edge cases**: Empty inputs, single elements, boundary values
33-
- **Corner cases**: Maximum/minimum constraints, special patterns
34-
- **Normal cases**: Typical scenarios with varied complexity
35-
- **Error cases**: Invalid inputs (if applicable)
36-
37-
### 3. Initial Validation
38-
39-
- Run `make p-test PROBLEM={problem_name}` to verify current implementation
40-
- **If errors found**:
41-
- DO NOT update implementation automatically
42-
- Only update test cases if they're incorrect
43-
- If implementation seems wrong, ASK USER first before modifying
44-
45-
### 4. JSON Template Update
46-
47-
- Update corresponding `.templates/leetcode/json/{problem_name}.json`
48-
- Add new test cases to `test_cases` field in proper format
49-
- Maintain existing test structure and naming conventions
50-
51-
### 5. Backup and Regeneration Process
52-
53-
- **Backup**: Move `leetcode/{problem_name}/` to `.cache/leetcode/{problem_name}/`
54-
- **Regenerate**: Run `make p-gen PROBLEM={problem_name} FORCE=1`
55-
- **Lint check**: Run `make p-lint PROBLEM={problem_name}`
56-
- **Iterate**: If lint fails, update JSON and regenerate until passes
57-
58-
### 6. Solution Preservation
59-
60-
- Copy `solution.py` from backup to newly generated structure
61-
- Run `make p-test PROBLEM={problem_name}` to verify tests pass
62-
- **If tests fail**: Go back to step 4, update JSON, and iterate until passes
63-
64-
### 7. Cleanup and Restore
65-
66-
- **CRITICAL**: Remove entire newly generated `leetcode/{problem_name}/` directory
67-
- **CRITICAL**: Restore original structure from `.cache/leetcode/{problem_name}/` backup
68-
- **CRITICAL**: Only THEN copy enhanced `test_solution.py` from generated files to restored structure
69-
- **CRITICAL**: Preserve existing solution class parametrization - if original test had multiple solution classes, restore them
70-
- Verify final state with `make p-test PROBLEM={problem_name}`
71-
- Clean up backup directory after successful verification
72-
73-
## Test Case Quality Standards
74-
75-
### Coverage Requirements
76-
77-
- **Minimum 10 test cases** per problem
78-
- **Total test cases after enhancement must exceed the threshold** used for detection (e.g., if threshold=10, final count must be >10)
79-
- **Edge cases**: 20-30% of total test cases
80-
- **Normal cases**: 50-60% of total test cases
81-
- **Corner cases**: 20-30% of total test cases
82-
83-
### Test Case Categories
84-
85-
#### Edge Cases
86-
87-
- Empty inputs: `[]`, `""`, `None`
88-
- Single element: `[1]`, `"a"`
89-
- Boundary values: `[0]`, `[1]`, `[-1]`
90-
- Maximum/minimum constraints from problem description
91-
92-
#### Corner Cases
93-
94-
- Duplicate elements: `[1,1,1]`
95-
- Sorted/reverse sorted arrays: `[1,2,3]`, `[3,2,1]`
96-
- All same elements: `[5,5,5,5]`
97-
- Alternating patterns: `[1,0,1,0]`
98-
99-
#### Normal Cases
100-
101-
- Mixed positive/negative numbers
102-
- Various array sizes within constraints
103-
- Different data patterns and structures
104-
- Representative problem scenarios
105-
106-
### JSON Format Requirements
107-
108-
- Use single quotes for Python strings in test cases
109-
- Follow existing parametrize format
110-
- Maintain type hints in parametrize_typed
111-
- Ensure test_cases string is valid Python list syntax
112-
- **NEVER include custom solution classes** in test_imports - only import the main solution class specified in solution_class_name
113-
- **PRESERVE existing solution class parametrization** - if original test had multiple solution classes, restore them after JSON regeneration
114-
115-
## Automation Scripts 🤖
116-
117-
**RECOMMENDED APPROACH**: Use prebuilt scripts in `.amazonq/rules/test-case-enhancement/scripts/` to streamline the enhancement process:
118-
119-
### Find Problems Needing Enhancement
12+
### 2. Enhancement Process
12013

12114
```bash
122-
# Find problems with ≤10 test cases (default)
123-
./.amazonq/rules/test-case-enhancement/scripts/find_problems_needing_enhancement.sh
124-
125-
# Custom threshold and max results
126-
./.amazonq/rules/test-case-enhancement/scripts/find_problems_needing_enhancement.sh 8 15
15+
# Simple 4-step process:
16+
# 1. Update JSON template with more test cases (12-15 total)
17+
# 2. Backup original
18+
mv leetcode/{problem_name} .cache/leetcode/{problem_name}
19+
# 3. Regenerate with enhanced tests
20+
make p-gen PROBLEM={problem_name} FORCE=1 && make p-lint PROBLEM={problem_name}
21+
# 4. Restore original solution, keep enhanced tests
22+
cp .cache/leetcode/{problem_name}/solution.py leetcode/{problem_name}/solution.py
12723
```
12824

129-
### Enhance Single Problem
25+
### 3. Verification
26+
27+
- Run `make p-test PROBLEM={problem_name}`
28+
- Fix any incorrect expected values in test cases
29+
- Update JSON template with corrections
30+
31+
### 4. Restore Backup
13032

13133
```bash
132-
# Enhance specific problem (handles full workflow automatically)
133-
./.amazonq/rules/test-case-enhancement/scripts/enhance_single_problem.sh <problem_name>
34+
# Copy enhanced test_solution.py to backup
35+
cp leetcode/{problem_name}/test_solution.py .cache/leetcode/{problem_name}/
36+
# Restore all original files (preserves user edits)
37+
rm -rf leetcode/{problem_name}
38+
mv .cache/leetcode/{problem_name} leetcode/{problem_name}
13439
```
13540

136-
### Batch Enhancement
41+
## Test Case Standards
13742

138-
```bash
139-
# Enhance up to 10 problems with ≤10 test cases (default)
140-
./.amazonq/rules/test-case-enhancement/scripts/enhance_batch_problems.sh
43+
### Coverage Requirements
14144

142-
# Custom batch size and threshold
143-
./.amazonq/rules/test-case-enhancement/scripts/enhance_batch_problems.sh 5 8
144-
```
45+
- **Minimum 12 test cases** per problem
46+
- **Edge cases**: Empty inputs, single elements, boundary values
47+
- **Corner cases**: Maximum/minimum constraints, duplicates, sorted arrays
48+
- **Normal cases**: Mixed scenarios with varied complexity
14549

146-
### Verify Enhancement Results
50+
### JSON Format
14751

148-
```bash
149-
# Verify enhancement was successful
150-
./.amazonq/rules/test-case-enhancement/scripts/verify_enhancement.sh <problem_name>
151-
```
52+
- Use single quotes for Python strings: `'hello'` not `"hello"`
53+
- Follow existing parametrize format
54+
- Ensure valid Python list syntax in test_cases field
15255

153-
## Manual Commands Reference
56+
## Quick Commands
15457

15558
```bash
156-
# Find problems needing more test cases
157-
poetry run python .templates/check_test_cases.py --threshold=10 --max=1
59+
# Find problems needing enhancement
60+
poetry run python .templates/check_test_cases.py --threshold=10
15861

15962
# Test specific problem
16063
make p-test PROBLEM={problem_name}
16164

16265
# Generate from JSON template
16366
make p-gen PROBLEM={problem_name} FORCE=1
16467

165-
# Lint specific problem
68+
# Lint check
16669
make p-lint PROBLEM={problem_name}
16770
```
16871

169-
## Error Handling
170-
171-
- **Implementation errors**: Ask user before modifying solution code
172-
- **Test failures**: Update JSON template and regenerate
173-
- **Lint failures**: Fix JSON format and iterate
174-
- **Backup failures**: Ensure `.cache/leetcode/` directory exists
175-
17672
## Success Criteria
17773

17874
- All tests pass with enhanced test cases
179-
- Total test cases exceed the threshold used for detection
180-
- Minimum 10 comprehensive test cases per problem
181-
- Original solution code preserved and working
75+
- Minimum 12 comprehensive test cases per problem
76+
- Original solution code preserved
77+
- **Enhanced test cases in final test_solution.py**
18278
- JSON template updated for future regeneration
183-
- Clean final state with no temporary files
184-
185-
## Automation Benefits
186-
187-
Using the prebuilt scripts provides several advantages:
188-
189-
### **Speed & Efficiency**
190-
191-
- **Single command execution** - No need to remember complex multi-step workflows
192-
- **Batch processing** - Enhance multiple problems simultaneously
193-
- **Automatic cleanup** - No manual file management required
194-
195-
### 🛡️ **Safety & Reliability**
196-
197-
- **Automatic backups** - Original code always preserved
198-
- **Error handling** - Graceful failure recovery
199-
- **Validation checks** - Syntax and structure verification
200-
201-
### 📊 **Visibility & Tracking**
202-
203-
- **Progress reporting** - Real-time status updates
204-
- **Result verification** - Automatic test case counting
205-
- **Success metrics** - Clear pass/fail indicators
206-
207-
### 🔄 **Consistency**
208-
209-
- **Standardized process** - Same workflow every time
210-
- **Quality assurance** - Built-in lint and validation checks
211-
- **Reproducible results** - Eliminates human error
212-
213-
**Recommendation**: Always use automation scripts for test case enhancement unless debugging specific issues requires manual intervention.

.amazonq/rules/test-case-enhancement/scripts/README.md

Lines changed: 0 additions & 76 deletions
This file was deleted.

0 commit comments

Comments
 (0)