-
-
Notifications
You must be signed in to change notification settings - Fork 11.5k
[Misc] Clean up more utils #27567
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Misc] Clean up more utils #27567
Conversation
Signed-off-by: DarkLight1337 <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request focuses on cleaning up and reorganizing utility functions within the vLLM project. It involves moving functions from vllm.utils to more specific submodules like vllm.utils.import_utils and vllm.utils.system_utils, removing unused code, and making minor adjustments to improve code clarity and maintainability. I have identified a critical issue where a function is being moved without updating its call site within the same file. This needs to be addressed to ensure the code functions correctly after the changes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💡 Codex Review
vllm/vllm/v1/worker/worker_base.py
Lines 187 to 194 in 88c0da5
| if vllm_config.model_config is not None: | |
| # it can be None in tests | |
| trust_remote_code = vllm_config.model_config.trust_remote_code | |
| if trust_remote_code: | |
| # note: lazy import to avoid importing torch before initializing | |
| from vllm.utils import init_cached_hf_modules | |
| init_cached_hf_modules() |
When trust_remote_code is enabled, the worker wrapper lazily executes from vllm.utils import init_cached_hf_modules. This helper was moved into vllm.utils.import_utils and is no longer exported from vllm.utils, so the import now raises ImportError and remote-code initialization fails. The lazy import should target the new module that defines init_cached_hf_modules.
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: DarkLight1337 <[email protected]>
| # Required for argparse hook only | ||
| -f https://download.pytorch.org/whl/cpu | ||
| cachetools | ||
| cloudpickle |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Strange that organising the utils would cause this to be required, is the mocking not working properly for this one?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Model executor (imported by docs) now imports serial_utils which in turn imports cloudpickle.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, these are relatively cheap to install, so it's not a big problem.
It'd be nice to figure out how to remove them in future.
Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: DarkLight1337 <[email protected]>
### What this PR does / why we need it? adapt vllm-ascend main branch with vllm releases/v0.11.1 fix `forward context not set` in test_vlm.py caused by: vllm-project/vllm#23207 fix import `cdiv round` failed caused by: vllm-project/vllm#27188 fix import `init_cached_hf_modules` failed caused by: vllm-project/vllm#27567 adapt triton kernel `fused_recurrent_gated_delta_rule_fwd_kernel` caused by: vllm-project/vllm#27654 - remove unused code in sigmoid_gating.py - `class FusedRecurrentFunction` , `fused_recurrent_gated_delta_rule`, `fused_recurrent_gated_delta_rule_fwd` ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? CI - vLLM version: v0.11.0 - vLLM main: vllm-project/vllm@83f478b Signed-off-by: 22dimensions <[email protected]>
### What this PR does / why we need it? adapt vllm-ascend main branch with vllm releases/v0.11.1 fix `forward context not set` in test_vlm.py caused by: vllm-project/vllm#23207 fix import `cdiv round` failed caused by: vllm-project/vllm#27188 fix import `init_cached_hf_modules` failed caused by: vllm-project/vllm#27567 adapt triton kernel `fused_recurrent_gated_delta_rule_fwd_kernel` caused by: vllm-project/vllm#27654 - remove unused code in sigmoid_gating.py - `class FusedRecurrentFunction` , `fused_recurrent_gated_delta_rule`, `fused_recurrent_gated_delta_rule_fwd` ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? CI - vLLM version: v0.11.0 - vLLM main: vllm-project/vllm@83f478b Signed-off-by: 22dimensions <[email protected]> Signed-off-by: luolun <[email protected]>
### What this PR does / why we need it? adapt vllm-ascend main branch with vllm releases/v0.11.1 fix `forward context not set` in test_vlm.py caused by: vllm-project/vllm#23207 fix import `cdiv round` failed caused by: vllm-project/vllm#27188 fix import `init_cached_hf_modules` failed caused by: vllm-project/vllm#27567 adapt triton kernel `fused_recurrent_gated_delta_rule_fwd_kernel` caused by: vllm-project/vllm#27654 - remove unused code in sigmoid_gating.py - `class FusedRecurrentFunction` , `fused_recurrent_gated_delta_rule`, `fused_recurrent_gated_delta_rule_fwd` ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? CI - vLLM version: v0.11.0 - vLLM main: vllm-project/vllm@83f478b Signed-off-by: 22dimensions <[email protected]> Signed-off-by: hwhaokun <[email protected]>
Purpose
Part of #26900
vllm.utils.init_cached_hf_modules -> vllm.utils.import_utils.init_cached_hf_modulesvllm.utils.import_pynvml -> vllm.utils.import_utils.import_pynvmlvllm.utils.get_mp_context -> vllm.utils.system_utils.get_mp_contextvllm.utils.kill_process_tree -> vllm.utils.system_utils.kill_process_treevllm.utils.set_ulimit -> vllm.utils.system_utils.set_ulimitvllm.utils.run_method -> vllm.v1.serial_utils.run_methodvllm.utils.check_use_alibi -> ModelConfig.uses_alibivllm.utils.enable_trace_function_call_for_thread -> VllmConfig.enable_trace_function_call_for_threadvllm.utils.argparse_utils.StoreBooleanis no longer used so I removed itTest Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.