docs: update router docs #2148
docs: update router docs #2148
Conversation
WalkthroughA new, detailed README for the KV Router component has been added, providing an overview, configuration, architecture, and tuning guidelines. An older, more narrowly focused KV Router performance tuning guide has been removed. No code or exported entities were changed. Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~3 minutes Possibly related PRs
Poem
Note ⚡️ Unit Test Generation is now available in beta!Learn more here, or try it out under "Finishing Touches" below. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
docs/components/router/README.md (2)
75-77: Add language specifiers to fenced code blocks
markdownlint(MD040) flags both fenced blocks because they omit an explicit language tag.
Specify something liketextto silence the linter and improve syntax highlighting:-``` +```text logit = kv_overlap_score_weight × potential_prefill_blocks + potential_active_blocks -``` +```…and similarly for the metric-logging example a few lines below.
Also applies to: 122-124
76-76: Parameter name is inconsistent (kv_overlap_score_weightvskv-overlap-score-weight)Line 76 uses snake-case (
kv_overlap_score_weight) while the surrounding sections and CLI flag examples use kebab-case (--kv-overlap-score-weight). Pick one style (ideally the exact CLI flag) and update all occurrences to avoid confusion.Also applies to: 85-85
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
docs/components/router/README.md(1 hunks)docs/guides/kv_router_perf_tuning.md(0 hunks)
💤 Files with no reviewable changes (1)
- docs/guides/kv_router_perf_tuning.md
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: PeaBrane
PR: ai-dynamo/dynamo#1285
File: lib/llm/src/kv_router/scheduler.rs:260-266
Timestamp: 2025-05-30T06:34:12.785Z
Learning: In the KV router scheduler code, PeaBrane prefers fail-fast behavior over silent failure handling. When accessing worker metrics data that could be out-of-bounds (like dp_rank indexing), explicit panics are preferred over graceful degradation with continue statements to ensure data integrity issues are caught early.
docs/components/router/README.md (1)
Learnt from: PeaBrane
PR: #1285
File: lib/llm/src/kv_router/scheduler.rs:260-266
Timestamp: 2025-05-30T06:34:12.785Z
Learning: In the KV router scheduler code, PeaBrane prefers fail-fast behavior over silent failure handling. When accessing worker metrics data that could be out-of-bounds (like dp_rank indexing), explicit panics are preferred over graceful degradation with continue statements to ensure data integrity issues are caught early.
🪛 markdownlint-cli2 (0.17.2)
docs/components/router/README.md
75-75: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
122-122: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Build and Test - vllm
Overview:
as titled
Summary by CodeRabbit