502 Errors for content disposition encoded in Base64

[alkampfergit]

I have intermittent error calling an api that download a document and I find that the error is the following one.

"ResponseHeaders": The destination returned a response that cannot be proxied back to the client.
System.InvalidOperationException: Invalid non-ASCII or control character in header: 0x00C2

Actually I have this exact response-header in original response.

Content-disposition: attachment; filename="=?utf-8?B?T0YwMTMxNjEgLSAxsCBMaW5lYQ==?="

It seems that YARP does not like RFC2045 encoding, actually decoding from Base-64 cyber chef reports valid UTF-8 base64 encoded

To Reproduce

Just proxy a request that returned the exact content-disposition header.

I'm using version 2.0.0

[MihaZupan]

YARP won't try decoding the filename, it'll copy the header back to the client as-is.

Invalid non-ASCII or control character in header: 0x00C2

It looks like the header value actually contained a non-ASCII character (that wasn't encoded in base64).
I can't repro the issue by using Content-disposition: attachment; filename="=?utf-8?B?T0YwMTMxNjEgLSAxsCBMaW5lYQ==?=" exactly, I suspect that the server response contained other characters.

You can try looking at the actual bytes the server responded with on the network, or a test transform like

.AddTransforms(builder =>
{
    builder.CopyResponseHeaders = false;

    builder.AddResponseTransform(context =>
    {
        if (context.ProxyResponse is not null)
        {
            if (context.ProxyResponse.Content.Headers.NonValidated.TryGetValues("Content-disposition", out var values))
            {
                Console.WriteLine($"Server sent 'Content-disposition: {values}' ({string.Join(", ", values.ToString().Select(c => (int)c))})");
            }
        }

        return default;
    });
});

---

And if you want to proxy non-ASCII header values, you can specify the encoding to use in all directions via

• KestrelServerOptions.RequestHeaderEncodingSelector for traffic between the client => proxy
• SocketsHttpHandler.RequestHeaderEncodingSelector for proxy => backend
• SocketsHttpHandler.ResponseHeaderEncodingSelector for proxy <= backend
• KestrelServerOptions.ResponseHeaderEncodingSelector for proxy <= client

See https://microsoft.github.io/reverse-proxy/articles/http-client-config.html#httpclient

[Tratcher]

0xC2 is Á. Was that in a different header?

[alkampfergit]

Actually it contains a file name that has some customer data so after my first post I've edited removing the sensitive part.

[MihaZupan]

Do you have any custom logic in your YARP app that processes the headers? Can you setup a minimal repro?

If it is the backend server generating this header, I can't reproduce any issues with proxying such a response through YARP.

[alkampfergit]

Managed to reproduce the issue https://github.com/alkampfergit/yarp-bug-reproduction

I have a ASP NET4.8 site that returns a file using this code (taken from base site)

protected HttpResponseMessage CreateStreamContentHttpResponseMessage(Stream fileStream, string filename)
{
	HttpResponseMessage httpResponseMessage = new HttpResponseMessage
	{
		Content = new StreamContent(fileStream)
	};

	httpResponseMessage.Content.Headers.Add("x-filename", filename);
	httpResponseMessage.Content.Headers.ContentType = new MediaTypeHeaderValue("text/plain");
	httpResponseMessage.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment")
	{
		FileName = filename
	};
	httpResponseMessage.Content.Headers.ContentLength = fileStream.Length;
	httpResponseMessage.StatusCode = HttpStatusCode.OK;
	return httpResponseMessage;
}

The problem is that the non-standard header called x-filename is returning a string that contains a NON ASCII character. I've mounted the site in localhost:9999 and download the file with http://localhost:9999/api/download from a browser and everything is ok.
If I download through YARP with http://localhost:5184/api/download I got the error. I didn't realize that the site is returning a custom x-filename header :( ....

It seems that the browser happily ignore that header, while YARP complains for non ASCII Charachter. I think that the bug can be closed.

[MihaZupan]

Note that you can proxy non-ASCII headers through YARP, doing so is just disabled by default.

You can specify the alternative encoding you want to use by tweaking SocketsHttpHandler.ResponseHeaderEncodingSelector and KestrelServerOptions.ResponseHeaderEncodingSelector in this case.

[ChintanRaval]

@MihaZupan thanks for the explanations here. We have the same requirement when using YARP. One of our downstream systems needs to return response header with UTF8 non-ASCII characters.

To do so, we understand that we would need to set both the below to the respective Encoding.

1. ✅ KestrelServerOptions.ResponseHeaderEncodingSelector
2. ❓ SocketsHttpHandler.ResponseHeaderEncodingSelector

Now, we were able to set the KestrelServerOptions.ResponseHeaderEncodingSelector by code such as below.

var builder = WebApplication.CreateBuilder(args);
builder.WebHost.ConfigureKestrel(kestrelServerOptions=>
{
    kestrelServerOptions.ResponseHeaderEncodingSelector = _ => System.Text.Encoding.UTF8;
});

However, how do we set the encoding for SocketsHttpHandler.ResponseHeaderEncodingSelector so that the HttpClient that YARP uses honours our encoding? As suggested, we had a look at HttpClientConfig, but found no such property there.

(thanks in advance :) )

[ChintanRaval]

ooh, I think you may have meant for consumers to use a Custom IForwarderHttpClientFactory.. we'll try that.