EFI: Express LTS Strategy

[wesleytodd]

Many decisions on which tools to use, coding patterns to follow, and generally run some technical decisions depend on how we decide to treat Node.js Version support and how to run major version releases. There is one requirement for an LTS strategy for this project:

1. We must maintain strict Node.js version support within a single express major

There are some really nice to haves:

1. Keeping our version support modern so we don't block node core from making progress
2. Keeping our version support modern so we don't get old outdated bug reports or security issues
3. Keeping our version support modern so we can adopt newer patterns faster

And then there are a lot of things to discuss around the cost and priorities of these things. I think we need to have this discussion, and I was waiting on things getting sorted more generally, but to keep the conversations focused I am opening this issue now so folks can centralize the conversation someplace without rehasing it in many different dependent issues like #184.

[ljharb]

Keeping our version support modern so we don't block node core from making progress

This merely requires continually testing on all supported versions of node, which both myself and pkgjs have actions to help you dynamically create a matrix for.

Keeping our version support modern so we don't get old outdated bug reports or security issues

I'm not sure what this means - as long as modern node is supported, a bug on an older supported version of node is still a bug.

Keeping our version support modern so we can adopt newer patterns faster

this really only applies to syntax, and that can be transpiled, so it needn't be a forcing function most of the time.

---

One of the inherent negative costs of any major bump is that people will be left behind on the old major, and backports for bug and security fixes become much more annoying and unlikely. When a major bump drops support for an engine, it makes upgrade the platform for that user much harder. I strongly suggest not arbitrarily adhering to someone else's support plan (LTS, eg) and instead, decide which node versions to support based on what minimal featureset is needed for express to support the user patterns it wants, and no fewer.

In the event that a dependency has a tighter contraint than express otherwise needs to impose, the tradeoffs should be ad-hoc weighed between letting that dependency make decisions for this project, vs the cost of finding an alternative dependency. (in most cases, i'll be happy to maintain an alternative dependency if that resolves this question)

[kibertoad]

this really only applies to syntax, and that can be transpiled

Transpilation is not the best approach for a framework, as that can cause unpredictable runtime overhead in some cases, especially when transpiling to a much older version. And it complicates debugging significantly.

[kibertoad]

@ljharb If the approach is to base supported version on desired libraries / syntax, it might make sense to start forming a wishlist for these. I'll start:

1. Modern testing library. My personal recommendation would be vitest, which requires node 18+, which is roughly similar to native Node test runner
2. Modern validation library. Typebox is Node 16+
3. Async/await (Node 8+)
4. Async iterators (Node 10+)
5. Optional chaining and nullish coalescence (Node 14+).

That to me looks like 18 would be great, and Node 14 would be a bare minimum.
What are the alternatives if Node 18 is too high of a bar? Minimalistic stuff like tap?

[ljharb]

You say “minimalistic” like it’s not one of the strongest arguments to consider a tool.

[kibertoad]

@ljharb It's subjective, of course, but I would take DX over minimalism any day. Adjustable height table that you control by a handle mechanically is minimalistic, but pressing the button to adjust the height is so much faster and more convenient.

If I can get away with not doing something by hand in coding with no runtime drawbacks, I will. And experience of using TAP in IDEs is so poor, I sometimes want to scream when debugging failing tests for fastify libs. And uncomfortable tooling = less outside contributors.

What are the advantages of minimalistic tooling? It's not even about learning curve, because sometimes it's even steeper as a result of having less convenience features available.

[ljharb]

Those were the same arguments that supported using jest, and the reasons most people are trying to move away from jest now are the same reasons I’d give.

[kibertoad]

@ljharb No, problem with Jest wasn't that it had poor DX. Actually, its API was so good, that both vitest and bun pretty much copied it verbatim, and it still hold up well. Problems with jest was that it
a) monkey-patched a lot of stuff that it shouldn't have
b) was abandoned by Meta

[ljharb]

Yes, I’m saying that the very good DX actually hid a lot of the complexity that turned out to be the root of the unavoidable problem. By avoiding that complexity, even at the cost of some DX, you save years of dev time later. My tape tests have never once needed any significant migration on 500 projects, over a decade ¯\_(ツ)_/¯

[kibertoad]

By avoiding that complexity, even at the cost of some DX, you save years of dev time later

Not necessarily. We've migrated our tests from Jest to Vitest without any changes, other than renaming jest. to vitest. and changing the config file. That's a one time migration effort, and then I'm saving time and energy every day.

It's somewhat strange to say that walking is superior to a car because you never need gasoline for it. That's technically true, but tradeoffs involved are way deeper than that.

[kibertoad]

My tape tests have never once needed any significant migration on 500 projects, over a decade

YMMV on that. I definitely had to do more changes to node-tap tests when bumping major versions in fastify codebase than I had to do to migrate from jest to vitest, or when bumping vitest semver major. That's a benefit of a stable, mature and well-designed API - it doesn't need to change.

[sheplu]

Changing a major version support (removing one) would be for me a breaking changed, and I agree to being tied to another project schedule release can be an issue.
But we should also work on lowering the need to support a large matrix of versions.

As posted there #172 it seems that a lot of other framework (and not all) adhere to something close to supporting the major version for the past 3 years which for us would make it between v14 and v16 if we were releasing a new major version today.

As this discussion is only for Express@5 (we won't change or deprecate versions supported by Express@4) we should discuss about that.
From what @kibertoad was also mentioning this would be quite close - and allow some "new" tooling

[ljharb]

If there is concrete difficulty in supporting a particular version ("having to run CI on it" doesn't count), or a concrete feature needed (the "exports" field, native ESM for some reason, async/await syntax, etc), then it makes perfect sense to constrain the supported versions accordingly. "it's old" is not a sensible reason, nor is "people who aren't us don't support it", is all.

[wesleytodd]

I think this conversation is a great example of where some agree upon technical priorities would help. I think my ideal is that we find a good happy medium. Yes that means we might drop node versions for sometimes "arbitrary" reasons but IMO we should value "predictable schedule" over "maximal version support" (phrasing might not be great, but the I hope that gets the point across). That doesn't mean we don't value version support, just that we want predictable timelines more than we want to support older versions.