[FEATURE]: Implement automatic rate limiting when all KV router workers are busy

[jorgeantonio21]

Feature request

Description

Currently, when all KV router workers are busy, the system continues to accept requests which can lead to:

• Increased latency for all requests
• Potential system overload
• Poor user experience during high load

Describe the problem you're encountering

Problem Statement

The HTTP service lacks automatic rate limiting mechanisms when the underlying KV router workers are at capacity. This results in no feedback to clients about service overload, with poor overall experience.

Proposed Solution

Implement a comprehensive rate limiting system that:

Queue Management: Add a configurable queue depth in the KV router scheduler
NATS Event System: Publish events when queue reaches maximum depth
HTTP Service Integration: Subscribe to "all workers busy" events and activate rate limiting
Automatic Recovery: Clear rate limits after configurable duration
Graceful Degradation: Service continues operating even if rate limiting fails

Describe alternatives you've tried

No response

[rmccorm4]

Hi @jorgeantonio21, support for router rejection was added by @PeaBrane here recently: #2465 (with your collaboration as well). Marking as closed, but feel free to re-open or open new issue if needed.