|
1 | 1 | # Test Case Enhancement Rules |
2 | 2 |
|
3 | | -## Assistant Workflow for Adding Comprehensive Test Cases |
| 3 | +## Simple Enhancement Workflow |
4 | 4 |
|
5 | | -When user requests to enhance test cases for a problem, the assistant will: |
| 5 | +When user requests test case enhancement: |
6 | 6 |
|
7 | | -### 1. Problem Resolution (Priority Order) |
| 7 | +### 1. Problem Resolution |
8 | 8 |
|
9 | | -- **FIRST**: Try to resolve from context - check active file path or user-provided problem name |
10 | | -- **SECOND**: If context resolution fails, THEN run `poetry run python .templates/check_test_cases.py --threshold=10 --max=1` to auto-detect 1 problem with <10 test cases |
11 | | -- **LAST**: If both above fail, ask user to explicitly specify problem name |
| 9 | +- Use active file context or user-provided problem name |
| 10 | +- If unclear, run: `poetry run python .templates/check_test_cases.py --threshold=10 --max=1` |
12 | 11 |
|
13 | | -### 2. Test Case Generation |
| 12 | +### 2. Enhancement Process |
14 | 13 |
|
15 | | -- Read `leetcode/{problem_name}/README.md` for problem understanding |
16 | | -- Analyze existing test cases in `leetcode/{problem_name}/tests.py` |
17 | | -- Generate comprehensive test cases covering: |
18 | | - - **Edge cases**: Empty inputs, single elements, boundary values |
19 | | - - **Corner cases**: Maximum/minimum constraints, special patterns |
20 | | - - **Normal cases**: Typical scenarios with varied complexity |
21 | | - - **Error cases**: Invalid inputs (if applicable) |
22 | | - |
23 | | -### 3. Initial Validation |
24 | | - |
25 | | -- Run `make p-test PROBLEM={problem_name}` to verify current implementation |
26 | | -- **If errors found**: |
27 | | - - DO NOT update implementation automatically |
28 | | - - Only update test cases if they're incorrect |
29 | | - - If implementation seems wrong, ASK USER first before modifying |
30 | | - |
31 | | -### 4. JSON Template Update |
32 | | - |
33 | | -- Update corresponding `.templates/leetcode/json/{problem_name}.json` |
34 | | -- Add new test cases to `test_cases` field in proper format |
35 | | -- Maintain existing test structure and naming conventions |
36 | | - |
37 | | -### 5. Backup and Regeneration Process |
38 | | - |
39 | | -- **Backup**: Move `leetcode/{problem_name}/` to `.cache/leetcode/{problem_name}/` |
40 | | -- **Regenerate**: Run `make p-gen PROBLEM={problem_name} FORCE=1` |
41 | | -- **Lint check**: Run `make p-lint PROBLEM={problem_name}` |
42 | | -- **Iterate**: If lint fails, update JSON and regenerate until passes |
| 14 | +```bash |
| 15 | +# Simple 4-step process: |
| 16 | +# 1. Update JSON template with more test cases (12-15 total) |
| 17 | +# 2. Backup original |
| 18 | +mv leetcode/{problem_name} .cache/leetcode/{problem_name} |
| 19 | +# 3. Regenerate with enhanced tests |
| 20 | +make p-gen PROBLEM={problem_name} FORCE=1 && make p-lint PROBLEM={problem_name} |
| 21 | +# 4. Restore original solution, keep enhanced tests |
| 22 | +cp .cache/leetcode/{problem_name}/solution.py leetcode/{problem_name}/solution.py |
| 23 | +``` |
43 | 24 |
|
44 | | -### 6. Solution Preservation |
| 25 | +### 3. Verification |
45 | 26 |
|
46 | | -- Copy `solution.py` from backup to newly generated structure |
47 | | -- Run `make p-test PROBLEM={problem_name}` to verify tests pass |
48 | | -- **If tests fail**: Go back to step 4, update JSON, and iterate until passes |
| 27 | +- Run `make p-test PROBLEM={problem_name}` |
| 28 | +- Fix any incorrect expected values in test cases |
| 29 | +- Update JSON template with corrections |
49 | 30 |
|
50 | | -### 7. Cleanup and Restore |
| 31 | +### 4. Restore Backup |
51 | 32 |
|
52 | | -- **CRITICAL**: Remove entire newly generated `leetcode/{problem_name}/` directory |
53 | | -- **CRITICAL**: Restore original structure from `.cache/leetcode/{problem_name}/` backup |
54 | | -- **CRITICAL**: Only THEN copy enhanced `test_solution.py` from generated files to restored structure |
55 | | -- **CRITICAL**: Preserve existing solution class parametrization - if original test had multiple solution classes, restore them |
56 | | -- Verify final state with `make p-test PROBLEM={problem_name}` |
57 | | -- Clean up backup directory after successful verification |
| 33 | +```bash |
| 34 | +# Copy enhanced test_solution.py to backup |
| 35 | +cp leetcode/{problem_name}/test_solution.py .cache/leetcode/{problem_name}/ |
| 36 | +# Restore all original files (preserves user edits) |
| 37 | +rm -rf leetcode/{problem_name} |
| 38 | +mv .cache/leetcode/{problem_name} leetcode/{problem_name} |
| 39 | +``` |
58 | 40 |
|
59 | | -## Test Case Quality Standards |
| 41 | +## Test Case Standards |
60 | 42 |
|
61 | 43 | ### Coverage Requirements |
62 | 44 |
|
63 | | -- **Minimum 10 test cases** per problem |
64 | | -- **Edge cases**: 20-30% of total test cases |
65 | | -- **Normal cases**: 50-60% of total test cases |
66 | | -- **Corner cases**: 20-30% of total test cases |
| 45 | +- **Minimum 12 test cases** per problem |
| 46 | +- **Edge cases**: Empty inputs, single elements, boundary values |
| 47 | +- **Corner cases**: Maximum/minimum constraints, duplicates, sorted arrays |
| 48 | +- **Normal cases**: Mixed scenarios with varied complexity |
67 | 49 |
|
68 | | -### Test Case Categories |
| 50 | +### JSON Format |
69 | 51 |
|
70 | | -#### Edge Cases |
71 | | - |
72 | | -- Empty inputs: `[]`, `""`, `None` |
73 | | -- Single element: `[1]`, `"a"` |
74 | | -- Boundary values: `[0]`, `[1]`, `[-1]` |
75 | | -- Maximum/minimum constraints from problem description |
76 | | - |
77 | | -#### Corner Cases |
78 | | - |
79 | | -- Duplicate elements: `[1,1,1]` |
80 | | -- Sorted/reverse sorted arrays: `[1,2,3]`, `[3,2,1]` |
81 | | -- All same elements: `[5,5,5,5]` |
82 | | -- Alternating patterns: `[1,0,1,0]` |
83 | | - |
84 | | -#### Normal Cases |
85 | | - |
86 | | -- Mixed positive/negative numbers |
87 | | -- Various array sizes within constraints |
88 | | -- Different data patterns and structures |
89 | | -- Representative problem scenarios |
90 | | - |
91 | | -### JSON Format Requirements |
92 | | - |
93 | | -- Use single quotes for Python strings in test cases |
| 52 | +- Use single quotes for Python strings: `'hello'` not `"hello"` |
94 | 53 | - Follow existing parametrize format |
95 | | -- Maintain type hints in parametrize_typed |
96 | | -- Ensure test_cases string is valid Python list syntax |
97 | | -- **NEVER include custom solution classes** in test_imports - only import the main solution class specified in solution_class_name |
98 | | -- **PRESERVE existing solution class parametrization** - if original test had multiple solution classes, restore them after JSON regeneration |
| 54 | +- Ensure valid Python list syntax in test_cases field |
99 | 55 |
|
100 | | -## Commands Reference |
| 56 | +## Quick Commands |
101 | 57 |
|
102 | 58 | ```bash |
103 | | -# Find problems needing more test cases |
104 | | -poetry run python .templates/check_test_cases.py --threshold=10 --max=1 |
| 59 | +# Find problems needing enhancement |
| 60 | +poetry run python .templates/check_test_cases.py --threshold=10 |
105 | 61 |
|
106 | 62 | # Test specific problem |
107 | 63 | make p-test PROBLEM={problem_name} |
108 | 64 |
|
109 | 65 | # Generate from JSON template |
110 | 66 | make p-gen PROBLEM={problem_name} FORCE=1 |
111 | 67 |
|
112 | | -# Lint specific problem |
| 68 | +# Lint check |
113 | 69 | make p-lint PROBLEM={problem_name} |
114 | 70 | ``` |
115 | 71 |
|
116 | | -## Error Handling |
117 | | - |
118 | | -- **Implementation errors**: Ask user before modifying solution code |
119 | | -- **Test failures**: Update JSON template and regenerate |
120 | | -- **Lint failures**: Fix JSON format and iterate |
121 | | -- **Backup failures**: Ensure `.cache/leetcode/` directory exists |
122 | | - |
123 | 72 | ## Success Criteria |
124 | 73 |
|
125 | 74 | - All tests pass with enhanced test cases |
126 | | -- Minimum 10 comprehensive test cases per problem |
127 | | -- Original solution code preserved and working |
| 75 | +- Minimum 12 comprehensive test cases per problem |
| 76 | +- Original solution code preserved |
| 77 | +- **Enhanced test cases in final test_solution.py** |
128 | 78 | - JSON template updated for future regeneration |
129 | | -- Clean final state with no temporary files |
0 commit comments