Add support for load-shedding.

[abizjak]

Add an option to enable load shedding for the service constructed for the connection.

Motivation

In the server we are implementing we want to load-shed when the service is overloaded, and we want to limit the number of concurrent requests per connection. We use tower::load_shed layer for load shedding, but this only applies to the entire service.

If we additionally apply the per-connection concurrency limit the issue is that this limit is typically hit first and so the service responds with "not ready". As a result the load-shed layer of the service is never reached meaning that requests just queue up.

Solution

The solution is to optionally add a load-shed layer to the per-connection service constructed by tonic.

[MintuJ]

Setting load_shed on the tower::ServiceBuilder like the example in https://docs.rs/tonic/latest/tonic/transport/struct.Server.html#method.layer will result in an internal error status code. This integration would help in returning a more meaningful error code resource_exhausted.

@abizjak did you get a chance to work on this issue?

[djc]

Let's keep the features here in alphabetic order?

[djc]

I think the majority of our messages are lowercase, so please match that here.