RFC 6265 for Cookies - remaining issues

[antonfirsov]

This is a follow-up on #26141. In #39250 and #39781 we implemented fixes for path and domain related problems reported in in the original issue. With this, we unblocked most users, however we can't claim RFC 6265 conformance yet, there are still many deviations and obsolete behaviors we need to investigate and change.

In many cases browsers also deviate from RFC. In these cases we need to make decisions about desired behavior before implementing changes.

The topics I identified so far:

1. Deal with deprecated attributes

Cookie properties belonging to attributes which are not required by RFC 6265:
Version, Port, Comment, CommentUri, Discard (... more?)

Currently Version and Port are being used to pick a value for CookieVariant as following (analysis by @CarnaViire):

• CookieVariant.Unknown - unused - was used in headers check (Set-Cookie vs Set-Cookie2) which is not actually executed now afaik
• CookieVariant.Plain - no version or explicit version=0
• CookieVariant.Rfc2109 - version=1 or higher and port property not set
• CookieVariant.Rfc2965 - port property is set (auto-assigns version to 1 as well)

The value is then being used to multiplex between different behaviors, where only CookieVariant.Plain path is RFC 6265-conformant. If the client sends cookies with obsolte attributes, our logic will deviate from the standard which may lead to issues similar to #19746 and #20942. Additionally, they obfuscate our code and tests a lot, creating technical debt.

Suggested action plan

• Investigate if presence of Version or Port does change browser behavior in any way. (Likely not.)
• Make a decision whether we should keep any of those old behaviors
• Ideally, the answer would be "no". In that case:
  • Remove CookieVariant and related logic. Note: the type is not public, it's not in the reference assemblies, but we should be careful to not break serialization compatiblity.
  • File an API proposal obsoleting all old properties

2. Further cases with leading dot in Domain

According to Section 5.2.3 of the RFC, clients should also include a logic to remove the leading dot when sent by Set-Cookie:

If the first character of the attribute-value string is %x2E ("."):
Let cookie-domain be the attribute-value without the leading %x2E
(".") character.

However browsers are typically ignoring this rule except for one-component domains. In some cases even adding a dot. From #39781 (comment):

Set-Cookie	CookieContainer	Chrome	Edge	RFC
localhost (implicit)	localhost	localhost	localhost	localhost (?)
localhost	localhost	localhost	localhost	localhost
.localhost	.localhost	localhost	cookie rejected	localhost
xample.com (implicit)	xample.com	xample.com	xample.com	xample.com (?)
xample.com	xample.com	.xample.com	xample.com	xample.com
.xample.com	.xample.com	.xample.com	.xample.com	xample.com

Note:
implicit means, that Set-Cookie did not set any value for domain, and a default value has been defined by the host name in the URL. I'm not sure if my current understanding is correct, but I see a chance browsers are actually non-conformant by defaulting the Domain attribute in this case. (See section 4.1.2.3 for more details.) This needs further clarification.

UPDATE: we decided to leave current behavior as-is.

Edit 2023: We should try to harmonize explicit/implicit domain cases like the one discovered in #84820.

3. Ordering cookies

Our cookie order doesn’t seem to match the one in browsers, and it likely doesn’t conform to the new RFC. We need to investigate this, and decide if we want to implement any changes.

/CC @CarnaViire @geoffkizer @karelz

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

[geoffkizer]

Re (2), I think it will be hard to change this behavior in the future. So we should make sure that we think what we are doing today is correct.

Re (1) and (3), I think we can deal with these in a future release.

[antonfirsov]

We should take into consideration Chromium's CookieMonster doc:

RFC 6265 is currently controlling, and supersedes RFC 2965. However, most browsers do not actually follow those RFCs, and Chromium has compatibility with existing browsers as a higher priority than RFC compliance. An RFC that more closely describes how browsers normally handles cookies is being considered by the RFC; it is available at http://tools.ietf.org/html/draft-ietf-httpstate-cookie.

Weirdly, the doc it references is an older draft of RFC 6265. The current WIP update of the cookie standard is here: https://datatracker.ietf.org/doc/draft-ietf-httpbis-rfc6265bis