WRT PR #3 - ES module path resolution

[bmeck]

This is the place to discuss specifics of how the EcmaScript (ES) module imports will resolve paths. The proposal itself can be seen in PR #3.

Discussions here should regard:

• What differences module resolution in ES module import syntax will have from require's module resolution algorithm.

It should not discuss import/export conversion, evaluation ordering, or detecting file mode.

[caridy]

To be clear, this is about import and export statements, whether they are declarative or imperative calls, it doesn't really matter because they are reflective somehow.

Additionally, we should, somehow, try to weight-in the resolution process used in browsers, to facilitate portability, I think @domenic can give us a quick overview of what we have so far in HTML.

Questions to be ask:

1. should the importer always include the file extension?
2. should we disallow the ambitious mechanism we do today in node to resolve index.js? specifically, the fact that require('./something') could resolve in ./something.js or ./something/index.js.
3. should we allow introspection into the module's internals? e.g.: import foo from "mod/lib/util"; or should we consider this a legacy behavior, and if you want to access the internals of the module, you should use require(), while an npm pkg will have a single es6 module that represents its api? (this somehow leaks into the discussion about .jsm vs package.json out of band signal)

[domenic]

What browsers are implementing is documented at https://html.spec.whatwg.org/#resolve-a-module-specifier. The answer to your questions for browsers are:

1. Yes, file extension is always necessary; import "./foo" looks for a file named foo, not one named foo.js.
2. Yes, index.js automagic is disallowed. import "./something" looks for a file named something, not one named something/index.js.
3. Not applicable yet, as any non-absolute-URL import specifiers not prefixed with /, ./, or ../ fail. Both seem possible in the future; when a mapping table API is created (manifest etc.) it could either be based on prefix-matching (allowing inspection of "internals") or just string equality (only allow inspection of "internals" if explicitly exposed in the manifest).

[bmeck]

The proposal currently has these specified. Currently the behavior is:

1. yes, importer must include file extension
2. this is disallowed if not looking for a package, but when searching for package main we use the current require algorithm.
3. currently this is allowed, it is used by various packages on npm and would be a hard sell. Currently the proposal does require file extensions for inner files, that would only affect file extension (keep that discussion contained to WRT PR #3 - ES module detection #13).

[caridy]

@domenic yes, 3 does not apply to the browser, although you could enforce that in the future via loader hooks.

@bmeck excellent, 1 and 2 seems to be very straight forward. As for 3, I think the various packages on npm today that are doing so are fat packages, and the transpiled version of their source are doing require() anyways. Once node supports ES modules, they will have to rework their pkgs a little bit, and should be fine to introduce this restriction. For years, we have been battling this, it is a refactor hazard for module authors, and signaling that to consumers of the modules by forcing them to use an alternative api to import those internals, seems like a good tradeoff here.

[bmeck]

@caridy banning inner files will cause some issues, but can probably smooth that over. Going to say we ban inner imports for packages.

[domenic]

I think it would be good to give people an opt-in way to expose their internals. I agree it's a hazard that they're automatically exposed, but for e.g. packages like especially or IIUC lodash it's an important part of their API.

[caridy]

@caridy banning inner files will cause some issues, but can probably smooth that over. Going to say we ban inner imports for packages.

@bmeck I'm not proposing banning them, obivously, inside your pkg you can do whatever you want since you can do ./path/to/relative/files.js. My proposal is to ban it when using import "packageName/fileName.js";

packages like especially or IIUC lodash it's an important part of their API.

@domenic Yes, we have few popular pkgs using that technique, and those will continue to work for people who are using require() to import them, but if you want to use an import statement, you will have to do something like: import {Math} from "especially"; which is itself a namespace produce by: export * as Math from './math.js'; in your index.js which will describe the API of the pkg.

[jkrems]

@caridy We have packages that include service clients and stubs to use for testing. We really don't want to load the service stubbing code into our production apps. Disallowing separate entry points sounds like a pretty big restriction. "You can keep using require" is another way of saying "if you want consistency in your code base and you're doing anything beyond toy examples, ignore import". Which would be a rather unfortunate outcome.

[bmeck]

@domenic would the reexport work if the loader does not evaluate any dependency that only has reexported bindings being imported (words @_@)?

// especially
export * as Math from './math.js';

// consumer
import {Math} from 'especially';

If we specify that especially does not evaluate since it has no direct bindings requested by consumer. Would that work? (me is rereading 262 since this would be a change it looks like, feel free to correct me).

[domenic]

I don't think there's any need to change the execution strategy. Reexporting would work I guess. Just kind of ugly.

[bmeck]

Forget what I said, side effects could mean the execution order needs preservation. We can't skip.

[caridy]

@caridy We have packages that include service clients and stubs to use for testing. We really don't want to load the service stubbing code into our production apps. Disallowing separate entry points sounds like a pretty big restriction.

@jkrems this is precisely what this discussion is for, to surface use cases, like yours. Can you elaborate more about the structure of your pkg? maybe a link that we can look at?

"You can keep using require" is another way of saying "if you want consistency in your code base and you're doing anything what toy examples, ignore import". Which would be a rather unfortunate outcome.

to reiterate, it is the consumer or existing pkgs using require() that will continue using require() for their convenience. If, as a consumer, you really want to use import, then you follow the new API provided by the author of the pkg.

[timoxley]

We have packages that include service clients and stubs to use for testing. We really don't want to load the service stubbing code into our production apps.

This is a use case I've hit and pondered with ES6 modules a few times. While it's possible to control what gets exported, there doesn't seem to be any way to conditionally prevent unneeded code from being loaded at all.

[bmeck]

@timoxley you will need to use an imperative function to load things conditionally:

// declare what can be exported:
export if_debug;

// conditionally load using require
let if_debug = IS_DEBUG ? require('foo') : null;

// conditionally load using System.loader.import
let if_debug = null;

// block completion using `await` , this would not affect require
// after this point if_debug is defined
await System.loader.import('foo').then(ns => if_debug = ns;)

[caridy]

additionally, once we settle on the api for the import() imperative api, you should be able to do something like:

export let if_debug = null;

if (something === true) {
    if_debug = await import('./something');
}

and that is not that far from what you do today in node.