A100: Client-side WRR slow start configuration #498

anuragagarwal561994 · 2025-05-31T04:49:04Z

No description provided.

ejona86 · 2025-06-02T14:20:30Z

anuragagarwal561994 · 2025-06-06T08:44:28Z

@dfawley did we get a chance to check this

atollena

This would be a nice addition (it's something we've had requests for internally at Datadog). Generally the approach sounds good to me. Ideally we would be able to support this mechanism for other load balancers supported by gRPC and Envoy (lr & rr). But given that lr & rr do not implement endpoint weights, it seems difficult to fit the feature in those balancers.

client-side-weighted-round-robin-slow-start.md

anuragagarwal561994 · 2025-06-06T18:55:08Z

@atollena I have addressed your comments regarding the proposal, let me know if I need to make any changes in it

anuragagarwal561994 · 2025-06-16T08:38:48Z

@ejona86 @atollena I have addressed the new comments, requesting review for any futher modifications needed

anuragagarwal561994 · 2025-06-19T15:02:46Z

@ejona86 can I start with the implementation of the feature?

anuragagarwal561994 · 2025-06-30T08:00:14Z

@ejona86 @atollena will it be possible to review the proposal this week so that I can begin with the implementation of the feature

atollena

It would be useful to see an implementation of this in one of the 3 languages. I wouldn't wait for the gRFC to be approved before starting, especially since the result can be used as a custom, private LB policy before it is merged to upstream-supported gRPC implementations.

client-side-weighted-round-robin-slow-start.md

atollena · 2025-07-01T11:24:13Z

client-side-weighted-round-robin-slow-start.md

+
+## Implementation
+
+This will be implemented in all languages C++, Java, and Go.


This is a good place to add the actual implementation of your choice when you have it ready (it can be a draft or closed PR that you can re-open once the gRFC has been approved).

I have added the implementation for Java, would love if you could review the changes :)

grpc/grpc-java#12200

client-side-weighted-round-robin-slow-start.md

anuragagarwal561994 · 2025-07-02T13:11:24Z

@atollena I have opened the respective MR at the envoy side as well, they will merge it once the proposal is approved at our end. For this proposal I just need to handle the case for the timing part and make the respective changes in the proposal once done I will assign for re-review.

anuragagarwal561994 · 2025-07-07T17:31:06Z

@atollena @ejona86 so I have created a rough implementation, not updated the test cases as of now.

I have also updated the proto on the envoy side, but to merge it we require to approve this proposal, then sync the proto may be in a separate MR.

How should we go about this?

atollena

👍

A100-client-side-weighted-round-robin-slow-start.md

dfawley

This mostly seems fine to me, I just have some nits. Also, since I don't see a slow start config in the envoy WRR policy proto, I'm curious if you've (or someone else has?) sent a proposal to add it there already.

A100-client-side-weighted-round-robin-slow-start.md

dfawley · 2025-07-17T21:28:27Z

A100-client-side-weighted-round-robin-slow-start.md

+
+## Metrics
+
+The following metric will be exposed to help monitor the slow start behavior:


How important is this metric to include? Should we consider not adding it until it's needed?

Ideally in most of the cases the slow_start_window will be applicable only for a short duration like 1-2 mins and if everything works well, it will not be called again.

I just thought that we can implement it in disabled mode, if someone wants to keeps a track of it.

We can decide this and I will make the respective changes in the proposal

anuragagarwal561994 · 2025-07-18T05:59:29Z

@dfawley envoyproxy/envoy#40090 this is the MR I have created at envoy's end to include the slow_start_config in proto.

However it requires this grpc proposal to be approved first before it is merged to the master.

Then we will need to sync across repo and then we are good to implement this feature. I have added sample MR with the java implementation as well, please do check it as well.

markdroth

Thanks for writing this up!

A100-client-side-weighted-round-robin-slow-start.md

Signed-off-by: anurag.ag <[email protected]>

Co-authored-by: Antoine Tollenaere <[email protected]>

Co-authored-by: Doug Fawley <[email protected]>

Signed-off-by: anurag.ag <[email protected]>

…osal. Adds links, clarifies slow start implementation details, and aligns with linked A24 proposal. Signed-off-by: anurag.ag <[email protected]>

anuragagarwal561994 · 2025-08-29T01:40:13Z

@markdroth I have made the respective changes, let me know if there are more changes required

markdroth

This is looking really good! Only a few comments remaining.

For future reference, please don't force-push to a PR after reviews have started, because that makes it very difficult for a reviewer to tell what's changed since their last review pass.

Please let me know if you have any questions. Thanks!

A100-client-side-weighted-round-robin-slow-start.md

A24-lb-policy-config.md

A100-client-side-weighted-round-robin-slow-start.md

…rmatting, and changes `aggression` type to `double`. Signed-off-by: anurag.ag <[email protected]>

… formatting fixes. Signed-off-by: anurag.ag <[email protected]>

markdroth

Looks great!

anuragagarwal561994 · 2025-08-29T19:36:04Z

@ejona86 @atollena @dfawley can you help me review and approve this proposal?

anuragagarwal561994 · 2025-09-08T07:08:41Z

@ejona86 @dfawley can you help review and merge the proposal?

dfawley

One concern (and a minor change will be needed if we can't keep the same envoy PR number):

A100-client-side-weighted-round-robin-slow-start.md

anuragagarwal561994 · 2025-09-16T04:13:03Z

@ejona86 all the reviewers has accepted the review, how do we go from here?

anuragagarwal561994 · 2025-10-09T14:53:23Z

@ejona86 @atollena @markdroth the envoy MR is merged can we now close this?

markdroth · 2025-10-09T16:18:58Z

Doug and I have approved, but Eric hasn't yet. We need to get his approval before merging.

anuragagarwal561994 · 2025-10-09T16:36:57Z

@ejona86 can you have a look into this?

anuragagarwal561994 · 2025-10-31T15:15:40Z

@ejona86 a quick follow up on the proposal, as the envoy side MR is also merged. I will be starting the implementations in the upcoming month now

Implements slow start functionality as described in [gRFC A100](grpc/proposal#498) I still need to do some work on tests and do an e2e experiment with xDS to ensure that things are working as expected, but the core implementation is in a state that can be reviewed. At this phase I'm mainly looking for feedback on the overall approach and design and in the test code because I am worried that I might be missing some test utility that would make testing with `time` easier. Pending Items before this is ready to be merged: - [ ] Add more unit tests for slow start config cases - [ ] Manually verify with an xDS experiment that slow start is working as expected Release Notes * weightedroundrobin: Implements slow start functionality as described in [gRFC A100](grpc/proposal#498) Signed-off-by: sotiris <[email protected]>

ejona86

Nicely written. My READY comment will probably cause some discussion. I see this as very close to ready.

A100-client-side-weighted-round-robin-slow-start.md

ejona86 · 2025-11-05T19:38:40Z

A100-client-side-weighted-round-robin-slow-start.md

+introduced to track when an endpoint transitioned to ready state. This timestamp is separate from the existing
+`non_empty_since` timestamp used for the blackout period.
+
+The `ready_since` timestamp is set when an endpoint transitions from a non-ready state (e.g., CONNECTING,


Re-starting slow start each time an endpoint becomes READY seems much more aggressive than Envoy, and doesn't seem to serve much purpose. I'd expect options more like:

When an endpoint is added (since UNHEALTHY endpoints are filtered, this will re-trigger slow start when it becomes HEALTHY; dunno if there's interactions with stateful session affinity plumbing). This seems very close to Envoy ("If no active healthcheck is configured per cluster, immediately upon joining the cluster.")

The first time an endpoint is READY, and the first time it goes READY after having been TRANSIENT_FAILURE. This seems pretty normal for gRPC

It would seem bad for MAX_CONNECTION_AGE to keep triggering slow-start, and MAX_CONNECTION_AGE will trigger CONNECTING.

Yes that makes sense, so should I add the second one? That slow start should trigger when an endpoint becomes ready for the first time or transitioning from TRANSIENT_FAILURE.

Will also have to make changes with respect to the variable name, something like added_since?

This will need input from @markdroth and @dfawley. We discussed it (on the 7th?) and realized it was in weak conflict with the blackout period behavior. I'm fine with only "fixing" this one, or changing both this and blackout period behavior. I also have a sneaky suspicion that Envoy implemented blackout period differently than we did. But overall there's several different views on the subject and we need to align.

ejona86 · 2025-11-05T21:13:29Z

A100-client-side-weighted-round-robin-slow-start.md

+
+Please note that this metric will start as experimental and off by default, and we will consider stabilizing it in the
+future.
+


There's no "Temporary environment variable protection" section. How will the implementation process go here? Seems the changes should be behind a variable until at least someone reports it is working.

I was referring to the current metrics implementations, they are turned off by default and if someone enables them via lets say opentelemetry they get enabled. There is no environment vairable, although this is not very well documented yet for how a user can enable these metrics. I also struggled and got to know about them when I was going through the code base in java

I'm not concerned about the metrics, but the feature in general. The missing section would normally have been here.

anuragagarwal561994 marked this pull request as draft May 31, 2025 04:49

anuragagarwal561994 mentioned this pull request May 31, 2025

Support for slow start with WRR Load Balancer Policy grpc/grpc-java#12087

Open

anuragagarwal561994 marked this pull request as ready for review May 31, 2025 05:03

ejona86 requested review from ejona86 and markdroth June 2, 2025 14:20

atollena reviewed Jun 6, 2025

View reviewed changes

client-side-weighted-round-robin-slow-start.md Outdated Show resolved Hide resolved

client-side-weighted-round-robin-slow-start.md Outdated Show resolved Hide resolved

client-side-weighted-round-robin-slow-start.md Outdated Show resolved Hide resolved

anuragagarwal561994 requested a review from atollena June 22, 2025 01:09

atollena reviewed Jul 1, 2025

View reviewed changes

anuragagarwal561994 changed the title ~~Client-side WRR slow start configuration~~ A100: Client-side WRR slow start configuration Jul 1, 2025

anuragagarwal561994 force-pushed the wrrslowstart branch 2 times, most recently from 10be215 to 43b016a Compare July 1, 2025 12:39

This was referenced Jul 1, 2025

proto: Adds slow start config to wrr proto message envoyproxy/envoy#40090

Merged

Slow Start Config support for Client Side WRR envoyproxy/envoy#40091

Closed

anuragagarwal561994 force-pushed the wrrslowstart branch from 0a3ea16 to e490283 Compare July 2, 2025 14:32

anuragagarwal561994 requested a review from atollena July 2, 2025 14:37

anuragagarwal561994 mentioned this pull request Jul 2, 2025

Implementing SlowStartConfig for WRR Load Balancing Policy grpc/grpc-java#12200

Draft

atollena approved these changes Jul 17, 2025

View reviewed changes

A100-client-side-weighted-round-robin-slow-start.md Outdated Show resolved Hide resolved

dfawley reviewed Jul 17, 2025

View reviewed changes

markdroth reviewed Jul 18, 2025

View reviewed changes

anuragagarwal561994 requested review from atollena and markdroth July 20, 2025 10:39

anuragagarwal561994 and others added 8 commits August 28, 2025 14:06

Updates proposal to include ready_since timestamp

c0f6592

Signed-off-by: anurag.ag <[email protected]>

Adds Java Implementation

3de4cfc

Signed-off-by: anurag.ag <[email protected]>

Fix typo, making offers as offer

ed6c9a7

Co-authored-by: Antoine Tollenaere <[email protected]>

Apply suggestions from code review

d92cf99

Co-authored-by: Doug Fawley <[email protected]>

Fixes min_weight_percent nit

15247c9

Updates proposal with slow start metrics and related references

1d363cd

Signed-off-by: anurag.ag <[email protected]>

Updates proposal with experimental metric details and formatting fixes

082fc33

Signed-off-by: anurag.ag <[email protected]>

Fixes formatting inconsistencies and updates terminology in A100 prop…

c346e9a

…osal. Adds links, clarifies slow start implementation details, and aligns with linked A24 proposal. Signed-off-by: anurag.ag <[email protected]>

anuragagarwal561994 force-pushed the wrrslowstart branch from e495afb to c346e9a Compare August 29, 2025 01:38

markdroth reviewed Aug 29, 2025

View reviewed changes

anuragagarwal561994 added 2 commits August 29, 2025 21:24

Aligns A100 proposal with related gRFCs, updates references, fixes fo…

82a4089

…rmatting, and changes `aggression` type to `double`. Signed-off-by: anurag.ag <[email protected]>

Updates A100 proposal with min_weight_percent type change and minor…

c17b3f6

… formatting fixes. Signed-off-by: anurag.ag <[email protected]>

markdroth approved these changes Aug 29, 2025

View reviewed changes

atollena approved these changes Sep 4, 2025

View reviewed changes

dfawley reviewed Sep 12, 2025

View reviewed changes

A100-client-side-weighted-round-robin-slow-start.md Show resolved Hide resolved

A100-client-side-weighted-round-robin-slow-start.md Show resolved Hide resolved

dfawley approved these changes Sep 15, 2025

View reviewed changes

markdroth mentioned this pull request Oct 28, 2025

Slow Start Support grpc/grpc#40858

Closed

davinci26 mentioned this pull request Nov 1, 2025

weightedroundrobin: Implements Slow Start (gRFC A100) [WiP] grpc/grpc-go#8689

Open

2 tasks

anuragagarwal561994 mentioned this pull request Nov 2, 2025

feat(loadbalancer): Add LoadBalancerType Client Side Weighted Round Robin envoyproxy/gateway#7407

Open

ejona86 reviewed Nov 5, 2025

View reviewed changes


		## Implementation

		This will be implemented in all languages C++, Java, and Go.


		## Metrics

		The following metric will be exposed to help monitor the slow start behavior:


		Please note that this metric will start as experimental and off by default, and we will consider stabilizing it in the
		future.

A100: Client-side WRR slow start configuration #498

Are you sure you want to change the base?

A100: Client-side WRR slow start configuration #498

Conversation

anuragagarwal561994 commented May 31, 2025

Uh oh!

ejona86 commented Jun 2, 2025

Uh oh!

anuragagarwal561994 commented Jun 6, 2025

Uh oh!

atollena left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

anuragagarwal561994 commented Jun 6, 2025

Uh oh!

anuragagarwal561994 commented Jun 16, 2025

Uh oh!

anuragagarwal561994 commented Jun 19, 2025

Uh oh!

anuragagarwal561994 commented Jun 30, 2025

Uh oh!

atollena left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

anuragagarwal561994 commented Jul 2, 2025

Uh oh!

anuragagarwal561994 commented Jul 7, 2025

Uh oh!

atollena left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dfawley left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anuragagarwal561994 commented Jul 18, 2025

Uh oh!

markdroth left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

anuragagarwal561994 commented Aug 29, 2025

Uh oh!

markdroth left a comment

Choose a reason for hiding this comment

Uh oh!

dfawley left a comment •

edited

Loading

anuragagarwal561994 Nov 6, 2025 •

edited

Loading