Skip to content

[EPIC] Revisiting HTTPS defaults in ASP.NET Core #41990

New issue
New issue
Closed
#42621
#42027
Closed
[EPIC] Revisiting HTTPS defaults in ASP.NET Core#41990
#42621
#42027
Assignees
DamianEdwards
Labels
EpicGroups multiple user stories. Can be grouped under a theme.area-authIncludes: Authn, Authz, OAuth, OIDC, Bearerarea-commandlinetoolsIncludes: Command line tools, dotnet-dev-certs, dotnet-user-jwts, and OpenAPIarea-networkingIncludes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractionsfeature-kestrelfeature-templates
Milestone
7.0-rc1
@DamianEdwards

Description

@DamianEdwards
DamianEdwards
opened on Jun 1, 2022

Revisiting HTTPS defaults in ASP.NET Core

Work item issues:

Scoped out of .NET 7

Relevant context issues:

Background

ASP.NET Core HTTPS dev certificate

The .NET SDK includes a command line tool, dotnet dev-certs, for managing the local certificate to use for ASP.NET Core HTTPS during local development.

Installation

The ASP.NET Core HTTPS dev certificate (aka dev cert) is a self-signed certificate installed as part of the dotnet first-run experience. The certificate is not trusted however, as that is an interactive process that requires user confirmation.

The dev cert can be manually managed (checked, exported, imported) using the dotnet dev-certs https command line tool.

Trusting the dev cert

The dev cert is self-signed, meaning it is not issued by a trusted root certification authority, and thus must be trusted in order for certain scenarios to succeed without warnings or indeed error messages. There are generally two scenarios that are impacted by dev cert trust issues:

  • Web browsers will not show the visited page of an HTTPS protected site if the HTTPS certificate is not trusted. Instead, an error page is displayed which the user must bypass with a number of clicks in order to see the page.
  • dotnet -> dotnet process communication using HttpClient when the API URL is protected with HTTPS. The experience when the dev cert is not trusted differs depending on the platform, but on Linux is generally completely fails with an exception.

In Visual Studio, when launching an ASP.NET Core project with an HTTPS binding (which is the default in all our templates), the user is prompted to trust the dev cert if VS detects it is not already trusted.

When not using Visual Studio (i.e. launching projects from the command line, VS Code, etc.), the dev cert will not be trusted unless the user explicitly takes action to trust it. On Windows and macOS, the user can run dotnet dev-certs https --trust to trust the certificate:

  • On Windows, this attempts to copy the dev cert from the user's "Personal" cert store, to the user's "Trusted Root Certification Authorities" cert store. Windows will forcibly show a UI dialog at this point to confirm that the user wants to trust the dev cert as a root authority.

  • On macOS, this invokes security add-trusted-cert -d -r trustRoot -k /library/Keychains/System.keychain <<certificate>> to add the dev cert as a trusted root authority (-r trustRoot) to the admin cert store (-d) (need to confirm why the admin store and not current user) in the system keychain (/library/Keychains/System.keychain)

  • On Linux, the tool does not currently support trusting the dev cert, but rather prints the following message directing uers to the documentation:

    Trusting the HTTPS development certificate was requested. Trusting the certificate on Linux distributions automatically is not supported. For instructions on how to manually trust the certificate on your Linux distribution, go to https://aka.ms/dev-certs-trust

ASP.NET Core Kestrel defaults

If no other address binding information is provided (e.g. via environment variables, command line arguments, configuration, etc.), ASP.NET Core's default built-in web server Kestrel, defaults to listening on http://localhost:5000. If a valid ASP.NET Core HTTPS dev certificate is detected during application start, in addition to the default HTTP address, Kestrel will also listen on https://localhost:5001 for HTTPS traffic, using the installed ASP.NET Core HTTPS dev certificate. This certificate is identified by a hard-coded certificate organization id (oid).

HTTP/2

Kestrel defaults to supporting HTTP/2 over TLS (HTTPS) only. This matches what web browsers support too. Launching an ASP.NET Core site without HTTPS will mean only HTTP 1.1 connections can be connected.

ASP.NET Core templates

All ASP.NET Core project templates currently default to configuring the project for HTTPS. This includes:

  • Configuring the "applicationUrl" property in the project's launch profile in Properties/launchSettings.json to "https://localhost:<<https_port>>;http://localhost:<<http_port>>" (the values for <<https_port>> and <<http_port>> are generated by the template when the project is created). This property is read by dotnet run and Visual Studio when launching the project, with the value being passed to the application on launch via the ASPNETCORE_URLS environment variable.

    In addition to passing the whole value to the launched application, Visual Studio & Visual Studio for Mac will launch a browser at the first URL in the semi-colon delimited value (assuming the "launchBrowser" proprty is set to true), so the order is significant.

  • Configuring the HTTPS redirection middleware via a call to app.UseHttpsRedirection(); in Program.cs

    Except for the empty web template

  • Configuring the HSTS middleware via a call to app.UseHsts(); in Program.cs

    Except for the empty and web API templates
    This behavior can be disabled by passing the --no-https template option at project creation time, e.g.:

$ dotnet new webapp --no-https -o WebAppHttpOnly

Note that templates that support the --auth option will ignore the --no-https option if it is supplied along with the --auth option set to any value other than "None" or "Windows", i.e. specifying the --auth option forces HTTPS to be configured in the new project. In Visual Studio, the checkbox to disable HTTPS at project creation time is hidden if any authentication option other than "None" or "Windows" is selected. In VS for Mac the HTTPS checkbox is still shown if an authentication type is selected, but unchecking the HTTPS checkbox does not actually disable HTTPS (the value is ignored).

Note that manually configuring a project using individual auth to run without HTTPS by changing the launch profile to only include an HTTP address works with no issues. There are no warnings displayed by the browser and registration and auth flow work without issue. In fact, if you register and login while HTTPS is enabled, and then disable HTTPS and run the site again, you will still be logged in, despite the auth cookie from the HTTPS-session being marked as secure only (tested on both Windows and Mac, should figure out why that is.)

The grpc template does not support the --no-https option and thus can only be created with HTTPS configured. This was a product decision rather than a technical one. gRPC works fine over HTTP/2 over HTTP if both client and server assume HTTP.

SameSite cookies, cookie policy, and authentication

ASP.NET Core implements features to manage cookies including their use with the SameSite standard supported by modern browsers. SameSite is intended to enhance the security of website cookies by allowing the cookie owner to limit when that cookie will be sent with requests depending on the origin domain. ASP.NET Core currently defaults to suppressing cookies marked as SameSiteMode.None in responses over non-secure connections. This is intended to ensure that cookies that expressly opt-out of SameSite restrictions are only ever sent over HTTPS connections, to mitigate the chances of them being intercepted for replay, etc. and matches the SameSite policy that modern web browsers enforce.

Most cookies written out by ASP.NET Core do so with a SameSite policy of SameSiteMode.Unspecified or SameSiteMode.Lax and a secure policy of CookieSecurePolicy.SameAsRequest, so they get returned over non-HTTPS connections with no problem. The cookies associated with some authentication flows however, default to a SameSite policy of SameSiteMode.None, as does the session cookie used for IdentityServer (which is used in our Blazor WebAssembly Hosted and SPA project templates). This means these authentication flows do not work by default when browsing the site over non-HTTPS connections.

Experience issues

macOS

  • The first-run experience successfully adds the dev cert to the store with no prompt or other user interaction, but it does not trust it (just like on Windows) and prints a message to the console with further details about how to trust the dev cert.
  • Every newly created ASP.NET Core application prompts for keychain access so that Kestrel can load the dev cert (dotnet vs. apphost) upon launch via dotnet run or dotnet watch. This is a fairly scary looking prompt and is required for every new web project created. The workaround is to configure the project to now use the apphost, or create the project without HTTPS support via the --no-https template option.
  • Web browsers don't trust the dev cert so browsing to an app using HTTPS via the dev cert will trigger the browser's insecure/untrusted site UX.
  • dotnet -> dotnet communication via HTTPS using the untrusted dev cert fails with exception: 
    Unhandled exception. System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
 ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid because of errors in the certificate chain: UntrustedRoot
   at System.Net.Security.SslStream.SendAuthResetSignal(ProtocolToken message, ExceptionDispatchInfo exception)
   at System.Net.Security.SslStream.CompleteHandshake(SslAuthenticationOptions sslAuthenticationOptions)
   at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter adapter, Boolean receiveFirst, Byte[] reAuthenticationData, Boolean isApm)
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(HttpRequestMessage request)
   at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   at Program.<Main>$(String[] args) in /Users/damian/src/local/WebApp9Client/Program.cs:line 4
   at Program.<Main>(String[] args)
damian@Damians-Mini WebApp9Client %
  • Trusting the dev cert can be done via the CLI using dotnet dev-certs https --trust but this prompts with another fairly scary looking prompt which adds to the friction of using .NET.
    • Note that trusting the cert via dotnet dev-certs https --trust on macOS will not configure Firefox to trust the dev cert, as it uses its own certificate store. This experience is equivalent to Firefox on Windows and Linux though.
    • On my M1 Mac Mini using a YubiKey configured as a SmartCard to enable PIN-based login, running dotnet dev-certs https --trust always results in an extra UI prompt that I can never successfully complete. The first time it's run the command line prompts for my SmartCard PIN and despite a message in the console saying the trust failed, the certificate is indeed trusted. 
      damian@Damians-Mini local % dotnet dev-certs https --trust --check
No valid certificate found.
damian@Damians-Mini local % sudo dotnet dev-certs https --trust
Trusting the HTTPS development certificate was requested. If the certificate is not already trusted we will run the following command:
'sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain <<certificate>>'
This command might prompt you for your password to install the certificate on the system keychain.
SecTrustSettingsSetTrustSettings: The authorization was canceled by the user.
There was an error trusting HTTPS developer certificate.
damian@Damians-Mini local % dotnet dev-certs https --trust --check
A trusted certificate was found: 3FE7B6BD1322F0ADFB02B13392FA6916A71601AF - CN=localhost - Valid from 2022-05-23 15:47:24Z to 2023-05-23 15:47:24Z - IsHttpsDevelopmentCertificate: true - IsExportable: true
damian@Damians-Mini local %

Linux

  • Browsers don't trust the dev cert and so browsing to an HTTPS hosted ASP.NET Core app on localhost yields a certificate warning page in the browser rather than the actual site.
  • dotnet -> dotnet communication via HTTPS using the untrusted dev cert fails with exception (same as above exception from macOS)
  • dotnet dev-certs https --trust does not work, instead printing a message pointing to docs on how to trust the dev cert manually.
  • Trusting dev cert requires manual steps to get browsers and dotnet -> dotnet comms to work, and requires at least OpenSSL 1.1.1h, which is later than included version in Ubuntu 20.04

WSL

Visual Studio's container and WSL tools today automatically export the ASP.NET Core HTTPS dev cert from the host machine and import it into the application container or WSL instance as part of the launch experience. As the dev cert is the same as used by the Windows host, browsers running on the host continue to trust it despite it being served from the container or WSL and thus no warnings or errors are shown. This results in a smooth default experience when using ASP.NET Core with HTTPS in Visual Studio with containers or WSL.

Outside of Visual Studio however is a different story. To remove the errors and warnings associated with the dev cert requires manually exporting the certificate (via dotnet dev-certs https) on the host OS and then configuring it within the context of where the app is running (i.e. container image or WSL). In the case of WSL the dotnet dev-certs-https tool could be updated to support configuring the dev-cert for WSL by compiling a self-contained, single exe, trimmed app to perform operations on the WSL distro (rough prototype).

MAUI

When doing development with a MAUI app project it's common to also have an ASP.NET Core project that hosts APIs that are called by the MAUI app. When this is done during the dev inner-loop on a platform other than the host OS (e.g. in an Android emulator on Windows) the MAUI app can have a difficult time connecting and calling the ASP.NET Core APIs successfully using the default HTTPS-based experience due to the following issues:

  • The ASP.NET Core HTTPS dev cert is not trusted by the platform the MAUI client is running on, e.g. Android or iOS emulator.
  • ASP.NET Core apps only listen on localhost by default, which means apps running on other machines (including MAUI apps running in emulators) cannot connect to the ASP.NET Core app running on the host.

While the HTTPS defaults of ASP.NET Core do contribute to these issues, this experience is outside the immediate scope of this effort and should be investigated separately as part of the MAUI development experience.

Proposed changes

The current experience issues create friction for developers trying to use ASP.NET Core, especially those outside of Visual Studio on Windows. We should revisit the idea that these experience issues are the best trade-off to make for the product (i.e. encouraging "secure" by default development practices vs. low friction getting started experiences). It's worth noting that no other major development platform (e.g. NodeJS, Go, Python, Ruby, etc.) has defaults that lead to these kinds of issues so early in the development experience. We've known about these issues for a while and even author our getting started content to explicitly avoid these issues by passing --no-https in the project creation step so as to avoid the reader having these issues when following along. But of course folks aren't always following these finely crafted tutorial experiences and will often just hit the issues head on, resulting in a poor impression.

.NET 7

Prioritized, complete in preview.6/preview.7:

De-prioritized (can revisit for .NET 8 if desired):

Separate HTTPS launch profile

Update templates so that (asuming the --no-https option is false) two launch profiles are configured in the project's launchSettings.json file:

  1. The first, named "HTTP" and configured with the ApplicationUrls property set to just a non-HTTPS address 
    "HTTP": {
  "commandName": "Project",
  "dotnetRunMessages": true,
  "launchBrowser": true,
  "applicationUrl": "http://localhost:5227",
  "environmentVariables": {
    "ASPNETCORE_ENVIRONMENT": "Development"
  }
}
  2. The second, named "HTTPS", and configured with the ApplicationUrls property set to the usual two addresses, the first being HTTPS, the second being HTTP: 
    "HTTPS": {
  "commandName": "Project",
  "dotnetRunMessages": true,
  "launchBrowser": true,
  "applicationUrl": "https://localhost:7227;http://localhost:5227",
  "environmentVariables": {
    "ASPNETCORE_ENVIRONMENT": "Development"
  }
}

When launching this project at the command line using dotnet run or dotnet watch, the default profile selected will be the first profile, which is not confiugred to use HTTPS, and as such the application will start without attempting to bind to HTTPS and thus not attempt to load the dev cert. The HTTPS profile can be selected at launch time by passing --launch-profile <launch-profile> to dotnet run, e.g.:

$ dotnet run --launch-profile HTTPS"

We can we make this experience simpler by introducing an option alias for the --launch-profile option, e.g.:

$ dotnet run -lp https

The project will continue to have the HTTPS-aware code present so the project is ready for use with HTTPS.

Folks using VS/VSforMac will continue to get the experience they do today with a single launch profile set to use both HTTPS and HTTP and launching directly into the HTTPS address.

Things potentially impacted by not launching with HTTPS by default:

  • Visual Studio & VS for Mac logic in that it currently launches using the first launch profile defined in launchSettings.json which would now not have an HTTPS address
  • The user-jwts tool (currently in development) which defaults to using the first configured HTTPS address in the target project's launchSettings.json file as the Audience field to the generated JWTs. If an HTTPS address is not found, the audience must be defined manually whenever creating a new JWT.
  • OAuth/OIDC auth template options, namely AzureAD and MS Identity Web. Do they support non-HTTPS localhost URLs?
  • IdentityServer in the SPA templates, can they work with non-HTTPS localhost URLs?

Updates to behavior of dotnet dev-certs https on macOS

Prototype changes can be found https://github.com/dotnet/aspnetcore/tree/javiercn/dev-certs-macos-update

Note that dotnet dev-certs https would still install the dev cert into the system keychain during CLI first-run (this doesn't require any prompts) so that apps running on .NET 6 or lower continue to have the current behavior. It would also install the dev cert into the new disk location for .NET 7 apps.

Add support for trusting the dev-cert on supported Linux distros

Do this basically: [Https] dotnet dev-certs --trust support on Linux dotnet/aspnetcore#32842

Prototype changes can be found https://github.com/dotnet/aspnetcore/tree/javiercn/dev-certs-linux-trust

  • Update dotnet dev-certs https --trust for configuring dev cert trust in OpenSSL, Chrome, Edge, and Firefox.
  • Required validation with popular distros:
    • Ubuntu
    • Fedora
    • Debian
    • CentOS
    • Open SUSE
  • Setup manual CTI test coverage for popular distros
    • Ubuntu
    • Fedora

Add a launch profile property that controls use of app host during project launch

Implement proposal in dotnet run of a newly created ASP.NET Core project on MacOS prompts for password to allow keychain access dotnet/sdk#22544

This is less important assuming the other proposed changes for macOS are implemented so can be deprioritized.

Metadata

Metadata

Assignees

Labels

EpicGroups multiple user stories. Can be grouped under a theme.area-authIncludes: Authn, Authz, OAuth, OIDC, Bearerarea-commandlinetoolsIncludes: Command line tools, dotnet-dev-certs, dotnet-user-jwts, and OpenAPIarea-networkingIncludes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractionsfeature-kestrelfeature-templates

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions