-
-
Notifications
You must be signed in to change notification settings - Fork 10.7k
Refactor error handling for multiple exceptions in preprocessing #15650
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor error handling for multiple exceptions in preprocessing #15650
Conversation
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
Signed-off-by: JasonZhu1313 <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice cleanup!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code looks good. It's clearly reducing duplication.
This touches on a larger needed cleanup here. As you can see from the duplication you removed, we map everything to the same HTTP response (400). That's not the best in all cases. We should be mapping different failures to different HTTP responses.
For example, I thinkRuntimeError that's caught in one part of this change should be a 500 response instead of the current 400.
I'm just sharing in case you'd be interested in further work in this area! Thanks for the patch.
…m-project#15650) Signed-off-by: JasonZhu1313 <[email protected]> Signed-off-by: xinyuxiao <[email protected]>
…m-project#15650) Signed-off-by: JasonZhu1313 <[email protected]> Signed-off-by: Louis Ulmer <[email protected]>
…m-project#15650) Signed-off-by: JasonZhu1313 <[email protected]>
…m-project#15650) Signed-off-by: JasonZhu1313 <[email protected]>
…m-project#15650) Signed-off-by: JasonZhu1313 <[email protected]> Signed-off-by: Mu Huai <[email protected]>
PR Description:
This PR refactors the error handling code to catch multiple exceptions (ValueError, TypeError, jinja2.TemplateError) in a single except block, reducing code repetition and improving readability. The same logging and error response logic is applied to all three exception types.
Benefits:
Test:
Error trace is caught properly and shown in server live log when I passed the wrong model that didn't match the --tool-call-parser hermes:
vllm serve llama_model_path --enable-auto-tool-choice --tool-call-parser hermes