[Model] Improve enable chunked_prefill & prefix_caching logic.

[noooop]

Improve all pooling task

• [Frontend][Doc][5/N] Improve all pooling task | Polish encode (pooling) api & Document. #25524
• [Model] Add num_cached_tokens for PoolingRequestOutput #27378
• [Model] Allow users to control skip reading cache per request. #28194
• [Model] Improve enable chunked_prefill & prefix_caching logic. #26623
• [Model][6/N] Improve all pooling task | Support chunked prefill with ALL pooling #27145
• [Model][8/N] Improve all pooling task | Deprecation as_reward_model. Extract hidden states prefer using new multi-vector retrieval API #26686
• [Doc][Last/N] Improve all pooling task | Refactor pooling-related documentation #27963

These PRs are mostly conflicting with each other, so combining them into a series would better inform reviewers about what happened. And what else needs to be done after that?

Purpose

Improve enable chunked_prefill & prefix_caching logic.

Address:

• [CI Failure] fix_test_auto_prefix_cache_support #26053 (comment)
• [Config] Clean up SchedulerConfig initialization #28665

Test Plan

pytest -s -vvv tests/test_config.py::test_is_chunked_prefill_supported
pytest -s -vvv tests/test_config.py::test_is_prefix_caching_supported
pytest -s -vvv tests/models/language/pooling/test_auto_prefix_cache_support.py

Test Result

pass

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[zhuohan123]

Why do we need a new class for bool? I think these kind of classes add mental overhead for no good reason. Will leave more comments tmrw.

[noooop]

Why do we need a new class for bool? I think these kind of classes add mental overhead for no good reason. Will leave more comments tmrw.

BoolWithReason is a great method that combines the reasons why a feature can't be enabled, making the code cleaner, improving readability and testability. Perhaps the name BoolWithReason isn't perfect, but we need similar functionality.

（This part of the code is a complete nightmare, and it keeps biting us. Let’s refactor it so we can all have a life again!）

[hmellor]

I agree, all the reasons should just be warnings

[noooop]

@DarkLight1337 @hmellor

A default vllm_config will be constructed here, including the creation of a default SchedulerConfig, and the following info log will be output, which is confusing.

For example, BAAI/bge-base-en does not support chunked prefill, so chunked prefill is not used by default.

vllm serve BAAI/bge-base-en

INFO 11-22 15:35:41 [config/scheduler.py:207] Chunked prefill is enabled with max_num_batched_tokens=2048.