History and navigation, excluding the navigation API

[domenic]

Description

This focus area is about session history and navigation between URLs. This would include:

• Foundational interoperability work on things like: what happens when navigating (especially in edge cases); how iframes impact the session history; when load/hashchange/popstate/etc. events fire; the sync vs. async nature of history traversals; what happens if code tries to go back and then forward and then navigate; how well about:/data:/blob: URLs work for navigation; scroll restoration; fragment navigation; etc.

• The existing navigation and history APIs, i.e. window.location and window.history.

This does not include the new navigation API, which I was advised in #170 should be separated.

Rationale

These areas are historically a large pain for web developers. @tbondwilkinson wrote up a vivid example; you can also see the HTML Standard's list of issues tagged "interop" and either "topic: navigation" or "topic: session history". This pain often spills over into broken behavior for users, e.g. pages not loading, the back button not working, scrolling being broken, or even jQuery's file upload code failing due to interaction with javascript: URL iframes. I suspect implementers who have worked in this area, and been on the frontlines of relevant bug triage, know how thorny an area this is (cc @annevk, @hsivonen, @smaug----, @natechapin). Getting interop on the foundations and existing APIs will lay the ground for the new navigation API to solve a lot of this developer pain.

For some signals, we have use counters such as ~30% of pages using history.replaceState(), ~20% of pages using sandboxed iframes, or of course the fact that 100% of page loads happen via navigation :).

Now is a good time to work on this project. On the specification side, within the month of October we plan to land whatwg/html#6315, which will resolve ~45 known spec issues with a definitive, principled behavior based on a coherent model. As part of that, and in the month or so after landing, we'll be increasing the test coverage for navigation and history, ensuring that all existing tests match the updated spec.

Specification

The following sections of the HTML Standard:

• "Loading web pages" excluding "Security infrastructure for Window, WindowProxy, and Location objects", "The WindowProxy exotic object", "Origin-keyed agent clusters", "Cross-origin opener policies", "Cross-origin embedder policies" (upcoming rewrite version has the same scope)
• "The iframe element"
• "The a element"
• "The base element"
• "Links" excepting the "Link types" subsection

Tests

• html/browsers/, excluding origin/origin-keyed-agent-clusters/, the-windowproxy-exotic-object/, and a few of the tests in the-window-object/ that are related to JavaScript bindings and not to actual navigation.

• html/semantics/embedded-content/the-iframe-element/

• html/semantics/text-level-semantics/the-a-element

• html/semantics/links excluding the linktypes/ subdirectory

• html/semantics/document-metadata/the-base-element/

Note that as part of the spec work mentioned above, over the rest of 2022 we plan to (a) add more coverage to html/browsers/; (b) aspirationally, do a general reorganization and renaming of existing tests of the html/browsers/ directory, since they are some of the oldest tests and thus often their paths don't make any sense or they have useless filenames like 006.html or traverse_the_history_5.html. This should help ensure we're starting from a good baseline, e.g. by fixing any failing-in-all-browsers tests before 2023 starts.

Important note: wpt.fyi is especially fragile for these sorts of tests, often giving incorrect failures. (I.e., failures that don't match what a user sees by loading the corresponding test in the browser on wpt.live.) See, e.g., web-platform-tests/wpt#33590 and web-platform-tests/wpt#23364 . Fixing this would probably need to be a priority for the wpt.fyi infrastructure team, to make this project feasible.

[foolip]

I see there are some tentative tests in html/semantics/embedded-content/the-iframe-element. What spec change/decision do these depend on?

[tbondwilkinson]

History and navigation, excluding the navigation API is P2 for Google's closure library.

"We have a goog.history API that could probably be simplified or obviated entirely. This is also a point of confusion for many developers."

[jgraham]

The tests in https://wpt.fyi/results/html/semantics/links/downloading-resources?label=master&label=experimental&aligned&view=subtest and https://wpt.fyi/results/html/semantics/links/hyperlink-auditing?label=master&label=experimental&aligned&view=subtest depend on <a ping>, which is explicitly optional, and so not suitable to include in Interop 2023.

[domenic]

I think they could be fixed to not do so, but if that doesn't happen by the time 2023 rolls around, then yeah, excluding them makes sense.

[foolip]

Based on discussion with @natechapin we'd also like to exclude these tests from the proposal:

• named-access-on-the-window-object/
• open-close/open-features-*

[jgraham]

I know that the test change period is over, but we spent some considerable time looking in detail at the tests here, and have some further feedback on limiting the scope in ways that we believe will make this focus area more acceptable and more helpful at promoting interop.

• We are assuming that all tests marked as tentative or optional are implicitly excluded until they are marked as non-tentative with a corresponding call-for-consensus issue.
• We think that all tests which depend on history.length which aren't currently passing in Blink & WebKit should be excluded. This because we believe that changing implementations to match those tests will on average make the interop situation worse, particularly in light of planned changes to gecko. In the future these tests should likely be updated.
• We believe the back-forward-cache tests should be excluded due to ongoing discussion about the level of standardisation required for the eviction heuristics.
• We think the iframe_loading_lazy_* tests should be excluded as the corresponding feature is lower priority and doesn't seem to represent a significant interop risk.
• cross-origin-top-navigation-with-user-activation-in-parent.window.html and cross-origin-top-navigation-without-user-activation.window.html need closer auditing; we think the tests may themselves be buggy.

Apologies again for the late feedback, but it seemed more helpful to provide it now when there's still some possibility to update the focus area before final decisions are made.

[chrishtr]

Apologies again for the late feedback, but it seemed more helpful to provide it now when there's still some possibility to update the focus area before final decisions are made.

Just wanted to quickly say: thank you for the super detailed and actionable comments! It's very helpful.

[foolip]

In the MDN short survey on APIs & JavaScript, "History API (window.history)" was selected by ~16% of survey takers, putting it in the middle third of options. (There is some uncertainty as with any survey data.)

[nairnandu]

Thank you for proposing History and navigation for inclusion in Interop 2023.

We wanted to let you know that this proposal was not selected to be part of Interop this year. We had many strong proposals, and could not accept them all. This should not be taken as a comment on the technology as a whole, and resubmitting a proposal for this feature as part of a future round of Interop would be welcome.

For an overview of our process, see the proposal selection summary. Thank you again for contributing to Interop 2023!

Posted on behalf of the Interop team.