HttpClient Performance Slow Compared to .NET 4.7

[narciero]

I have conducted a basic performance test of HttpClient in .NET Core 2.0 versus HttpClient in .NET Framework 4.7 and have noticed a gap in performance. The .NET Framework 4.7 HttpClient/WebClient outperforms .NET Core's HttpClient by ~1.5-2x in terms of how long it takes to complete a batch of n requests.

Testing

The test is a console app (run in release mode on Windows 7 16GB 3.5GHz) with identical code for .NET Core/Framework that follows this sequence:

1. Create a single, shared HttpClient instance with a maximum of n (for testing n=10,100) connections.

// .NET Core 2.0
var httpClient = new HttpClient(new HttpClientHandler { MaxConnectionsPerServer = 100 });

// .NET Framework 4.7
ServicePointManager.DefaultConnectionLimit = 100;
var httpClient = new HttpClient();

// .NET Framework 4.7 - WebClient instance is created ONCE PER REQUEST
ServicePointManager.DefaultConnectionLimit = 100;
var webClient = new WebClient();

2. Start 10,000 simulataneous requests and time how long each request takes to complete + how long all take to complete. Here is code demonstrating how requests are made / timed.

private void RunTest(int count)
{
    // requests is a list of Request data structures used to store individual request latency / responses
    var requests = Enumerable.Range(0, count).Select(j => CreateRequest(j)).ToList();
    
    // this stopwatch is for duration of all requests
    var stopwatch = Stopwatch.StartNew();
    
    // start all requests and wait for completion
    var requestTasks = requests.Select(MakeRequest).ToArray();
    Task.WaitAll(requestTasks);
    
    stopwatch.Stop();
    // total run time = stopwatch.ElapsedMilliseconds
}

private Task MakeRequest(Request request)
{
    var stopwatch = Stopwatch.StartNew();
    var response = await httpClient.GetStringAsync(request.Url);
    stopwatch.Stop();
    
    // save request duration and response
    request.DurationMs = stopwatch.ElapsedMilliseconds;
    request.ResponseId = ParseResponse(response);
}

I am testing against a basic python server that is well under capacity to rule out any server side bottlenecks. The server returns a simple JSON response containing an id field which is used to validate the HttpClient responses.

{ "id": "some_identifier" }

JSON deserialization / response validation is NOT included in performance stats.

Results

These are average statistics (in milliseconds) of 5 separate runs of 10,000 requests each. It is worth mentioning that the actual time spent on each request was much less with .NET Core's HttpClient, it's just that the batch of requests takes longer to complete as a whole vs .NET Framework.

framework	total run time (ms)	avg req time (ms)	median req time (ms)	total time spent on requests (s)
.NET Core 2.0	5614	216	282	777
.NET 4.7	3055	1355	1339	10,585

[MarinAtanasov]

Frankly, I am surprised that you could do 10000 requests with this code.

I've tried something similar and got the same error as this person.

Going back to performance, I've tried testing AspNet Core 2.0's REST performance using a DotNet Core 2.0 console application (yes, I know that there are tools for that) and the client ended up being the bottleneck for my little test. I was expecting to be able to create more requests than the server can respond to. If I create too many requests at the same time, I get the linked error on the client. If I create them sequentially in several threads, the client's process has a much higher CPU usage than the server.

[narciero]

@MarinAtanasov I was initially receiving that error as well. Setting MaxConnectionsPerServer fixed the problem for me, but yes the client was still the bottleneck.

[MarinAtanasov]

Considering that there are tools which can measure upwards of a million requests per second (written in other languages), what are the max requests per second which can be achieved through a .Net Core 2.0 client and how much CPU/memory would it consume?

@narciero, from your test it would seem that you cannot do more than two thousand requests / second. Am I missing something obvious here?

[narciero]

@MarinAtanasov its important to note that i tested using MaxConnectionsPerServer set to 100, but yes under those conditions 2k req/s is what i observed.

[Priya91]

cc @karelz @wfurt Performance of HttpClient, is this something that can be measured in the new lab setup.

[wfurt]

The performance will depend on may factors - like using keep-alive, DNS, etc.
It is interesting that average request time for 2.0 is much shorter but overall duration is longer.
That may be because the time is spent in underlying native call. But that is hard to know for sure without more details.

I don't have any benchmarks for .net framework yet but it is on TODO list.

[karelz]

Yes, this is very much on our list for ManagedHandler. We plan to compare Windows vs. Linux vs. Desktop vs. other platforms vs. old HttpClientHandler.
We will need to measure more characteristics than just this - different payload sizes, number of concurrent requests, measure CPU and network utilization, try scale up (1 vs. 2 vs. 4 vs. 8 cores), etc.

Given that it is on our backlog and the data above is not complete and will need to be re-measured anyway, I don't think we need this issue opened. Closing (let me know if you think it is wrong).

[TechnikEmpire]

Do all similar objects suffer from this or is it just HttpClient?

[karelz]

This particular issue lacks details and deeper analysis. The results are a bit suspicious and were not attempted to reproduce by anyone else. There is a suspicion that the compared platforms don't use the same settings under the hood (e.g. MaxConnectionsPerServer is set differently on each platform). Also, other perf comparisons typically show better performance on .NET Core than on .NET Framework.
It is premature to claim that HttpClient "suffers from this" as we do not know at all what "this" is. At this moment I consider it a test issue, until proven otherwise.

[MarinAtanasov]

I am not sure how it is compared to .NET Framework, but we definitely know that HttpClient in .NET Core is incredibly slow and heavy. That is proven by the fact that we can expect performance improvements up to 1000% (x10) in .NET Core 2.1. Source: .NET Core 2.1 Roadmap.

@TechnikEmpire, if you are using microservices, expect this to be a bottleneck. We had a framework performance comparison for one of our microservices where .NET Core had about 50% of the performance of Node or Go. We should get our hands on .NET Core 2.1 sometime during the summer. Until then, I do not know if there are any fast alternatives to using HttpClient.

[TechnikEmpire]

@MarinAtanasov I'm using the client to handle the upstream side of a local transparent filtering proxy and yeah it shows. I guess I'll just target .net full for now to avoid the issue.

[karelz]

@MarinAtanasov do you have any evidence that HttpClient in .NET Core is incredibly slow and heavy? Are you measuring Windows or Linux?
While we were able to speed up specific scenarios (esp. on Linux) in 2.1 (up to 10x), I would not use it as a "proof" that HttpClient in 2.0 is slow in all scenarios.

@TechnikEmpire if you have a specific code which shows performance difference between .NET Framework and .NET Core - especially on Windows, we would be very interested in getting our hands on it. Is it something you could share as a perf test?

[TechnikEmpire]

@karelz I'll see if I can't whip up a test case. I test my proxy code by running an external Apache tool to measure latency and throughput so I'm wondering if you'll accept such a test case.

[wfurt]

That may be ok @TechnikEmpire as well as the original post simply measured total time to completion.
In ideal case we would get simple console app + instructions how to run it with external dependencies.
It should be full app one can look at and run, not just code fragments. Ability to reproduce and verify claims easily will greatly improve chance this will be addressed.