JACoB PR for Issue Enhance the Jira to GitHub issue conversion

[jacob-local-kevin]

Summary:

Summary

Many incoming Jira tickets lack sufficient detail for JACoB to process them effectively. This leads to wasted resources as we create plans and research items for tasks that cannot be completed due to inadequate information. To improve efficiency, we need to implement an evaluation step before converting Jira issues into GitHub issues.

Proposed Solution

Enhance the Jira to GitHub issue conversion process by introducing an issue quality evaluation step. Instead of always rewriting the Jira title and description and creating a new GitHub issue, we will first assess the quality of the Jira issue to determine if it contains enough detail for JACoB to handle.

Implementation Steps

1. Implement evaluateJiraIssue Function

• File: /src/server/utils/evaluateIssue.ts
• Task:
  • Add a new function evaluateJiraIssue that:
    • Accepts the title and description of a Jira issue as inputs.
    • Returns an evaluation score between 1 and 5 (half-points acceptable), indicating how likely it is that an AI coding agent can flawlessly complete the task.
    • Provides a one-sentence feedback message if the score is less than 4, informing the user what needs to be changed to make the ticket actionable by JACoB.
  • Note: Define clear evaluation criteria for scoring to ensure consistent assessments.

2. Update the Jira Issue Conversion Process

• File: /src/server/utils/jira.ts
• Task:
  • Modify the convertJiraIssueToGithubIssue function to include the evaluation step:
    • Before creating a GitHub issue, call the evaluateJiraIssue function with the Jira issue's title and description.
    • If the evaluation score is less than 4:
      • Do not create a GitHub issue.
      • Save the evaluation score, feedback message, and Jira issue description in the database.
    • If the evaluation score is 4 or higher:
      • Proceed with rewriting the Jira issue and creating a GitHub issue as usual.

3. Update the Issues Database Schema

• File: Create migration script /src/server/db/migrations/20241130000000_updateIssuesTable.ts
• Task:
  • Modify the issues table to include the following optional fields:
    • jiraIssueDescription: text field to store the original description from the Jira issue.
    • evaluationScore: numeric field to store the score from evaluateJiraIssue.
    • feedback: text field to store the feedback message provided to the user.
    • didCreateGithubIssue: boolean field (default false) indicating whether a GitHub issue was created.
  • Ensure the migration handles existing data appropriately to maintain data integrity.

4. Communicate Feedback to Users

• Task:
  • User Interface Update:
    • Update the UI components in the Jira integration (e.g., issue status displays) to show the feedback message when a Jira issue is not converted.
  • Alternative Notification:
    • Optionally, implement a feature to send a notification or comment directly on the Jira issue with the feedback message.
  • Note: Clearly specify in documentation or code comments how the feedback is communicated to users.

5. Update Tests and Documentation

• Task:
  • Testing:
    • Update existing unit tests or create new tests to cover:
      • The evaluateJiraIssue function for correct scoring and messaging.
      • The updated convertJiraIssueToGithubIssue function for both paths (issue created and not created).
  • Documentation:
    • Update relevant documentation to reflect the changes made to the issue conversion process.
    • Include guidelines on the evaluation criteria used by evaluateJiraIssue.

Expected Outcome

By implementing this evaluation step in the Jira to GitHub issue conversion process:

• Improved Efficiency: JACoB will focus on Jira tickets that have sufficient detail, improving resource utilization.
• Resource Optimization: Resources will no longer be wasted on creating plans and research items for incomplete or vague tickets.
• User Feedback: Users will receive immediate feedback on how to improve their Jira tickets to make them actionable by JACoB.
• Enhanced AI Effectiveness: The overall effectiveness of the AI coding agent will be enhanced as it works with well-defined tasks.
  @jacob-ai-bot --skip-build

Plan:

Step 1: Edit /src/server/utils/evaluateIssue.ts

Task: Implement evaluateJiraIssue function

Instructions:
In the file /src/server/utils/evaluateIssue.ts, add a new function evaluateJiraIssue that accepts the title and description of a Jira issue as inputs. The function should return an evaluation score between 1 and 5 (allowing half-points), indicating how likely it is that an AI coding agent can flawlessly complete the task. If the score is less than 4, the function should also provide a one-sentence feedback message informing the user what needs to be changed to make the ticket actionable by JACoB. Define clear evaluation criteria for scoring to ensure consistent assessments.

Exit Criteria:
The evaluateJiraIssue function is implemented, tested, and returns appropriate evaluation scores and feedback messages based on the input title and description.

Step 2: Edit /src/server/utils/jira.ts

Task: Update Jira issue conversion process to include evaluation step

Instructions:
In the file /src/server/utils/jira.ts, modify the fetchNewJiraIssues function to include the evaluation step before creating a GitHub issue. Before calling rewriteGitHubIssue, call the evaluateJiraIssue function with the Jira issue's title and description. If the evaluation score is less than 4, do not create a GitHub issue. Instead, save the evaluation score, feedback message, and Jira issue description in the database. If the score is 4 or higher, proceed with rewriting the Jira issue and creating a GitHub issue as usual.

Exit Criteria:
The Jira issue conversion process includes the evaluation step, and behaves correctly based on the evaluation score (i.e., only creates GitHub issues for scores >= 4, otherwise saves the evaluation data).

Step 3: Create /src/server/db/migrations/20241130000000_updateIssuesTable.ts

Task: Create migration script to update issues table

Instructions:
Create a new migration script /src/server/db/migrations/20241130000000_updateIssuesTable.ts to modify the issues table schema. Add the following optional fields to the issues table: jiraIssueDescription (text), evaluationScore (numeric), feedback (text), and didCreateGithubIssue (boolean, default false). Ensure the migration handles existing data appropriately to maintain data integrity.

Exit Criteria:
The migration script is created and adds the new fields to the issues table without affecting existing data.

Step 4: Edit /src/server/db/tables/issues.table.ts

Task: Update IssuesTable schema to include new fields

Instructions:
In the file /src/server/db/tables/issues.table.ts, update the IssuesTable class to include the new columns added in the migration script. Add the following fields to the columns definition: jiraIssueDescription (t.text().nullable()), evaluationScore (t.numeric().nullable()), feedback (t.text().nullable()), and didCreateGithubIssue (t.boolean().default(false)).

Exit Criteria:
The IssuesTable schema is updated to include the new fields, matching the database schema.

Step 5: Edit /src/app/dashboard/[org]/[repo]/todos/Todo.tsx

Task: Display evaluation feedback in the UI for Jira issues

Instructions:
In the file /src/app/dashboard/[org]/[repo]/todos/Todo.tsx, modify the component to display the feedback message when a Jira issue is not converted into a GitHub issue due to a low evaluation score. The feedback message should be displayed alongside the issue in the UI, informing the user about what needs to be changed to make the ticket actionable by JACoB.

Exit Criteria:
The UI component displays the feedback message for Jira issues that were not converted into GitHub issues, providing users with actionable information.

Step 6: Edit /src/server/utils/evaluateIssue.test.ts

Task: Add unit tests for evaluateJiraIssue function

Instructions:
In the file /src/server/utils/evaluateIssue.test.ts, add new unit tests for the evaluateJiraIssue function. Cover various scenarios with different input titles and descriptions, ensuring the function returns the correct evaluation scores and feedback messages according to the defined criteria.

Exit Criteria:
Unit tests for evaluateJiraIssue are implemented, covering multiple scenarios, and all tests pass.

Step 7: Create /src/server/utils/jira.test.ts

Task: Create tests for updated Jira issue conversion process

Instructions:
Create a new test file /src/server/utils/jira.test.ts to add tests for the updated Jira issue conversion process. Write tests to verify the behavior when the evaluation score is less than 4 (ensuring that no GitHub issue is created and the evaluation data is saved) and when the score is 4 or higher (ensuring that a GitHub issue is created). Mock necessary dependencies and handle API calls appropriately.

Exit Criteria:
Tests for the Jira issue conversion process are implemented, covering both cases based on evaluation score, and all tests pass.