Windows-specific `ERR_UNKNOWN_MODULE_FORMAT`

[brianjenkins94]

Possibly related:

• HTTPS imports fail for relative URLs nodejs/node#42098
• nodejs/node /lib/internal/modules/esm/loader.js

Search Terms

• npx

Context

I run my typescript (served via npx) through ts-node so I don't have to maintain a build.

Additional Context: #1663

Expected Behavior

The program runs without error. In the case of the linked gist, it runs hello.ts, printing "Hello, world!".

Actual Behavior

On Windows (CMD or Git Bash, doesn't matter)

For some reason the error gets swallowed:

$ npx https://gist.github.com/brianjenkins94/d73578d7439427a59df28c1bc751e02d

npm WARN exec The following package was not found and will be installed: gist:d73578d7439427a59df28c1bc751e02d

> node --experimental-specifier-resolution=node --loader=file:///C:/Users/User/AppData/Local/npm-cache/_npx/caa40e90fbe6bfad/node_modules/ts-node/esm.mjs C:/Users/User/AppData/Local/npm-cache/_npx/caa40e90fbe6bfad/node_modules/sampleThatReproducesTheIssue/hello.ts

So I copy and paste what it was supposed to spawn:

$ node --experimental-specifier-resolution=node --loader=file:///C:/Users/User/AppData/Local/npm-cache/_npx/caa40e90fbe6bfad/node_modules/ts-node/esm.mjs C:/Users/User/AppData/Local/npm-cache/_npx/caa40e90fbe6bfad/node_modules/sampleThatReproducesTheIssue/hello.ts
(node:22652) ExperimentalWarning: --experimental-loader is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:22652) ExperimentalWarning: The Node.js specifier resolution in ESM is experimental.
(node:22652) ExperimentalWarning: The Node.js specifier resolution in ESM is experimental.
RangeError [ERR_UNKNOWN_MODULE_FORMAT]: Unknown module format: null for URL file:///C:/Users/User/AppData/Local/npm-cache/_npx/caa40e90fbe6bfad/node_modules/sampleThatReproducesTheIssue/hello.ts
    at new NodeError (node:internal/errors:372:5)
    at ESMLoader.load (node:internal/modules/esm/loader:384:13)
    at async ESMLoader.moduleProvider (node:internal/modules/esm/loader:282:11)
    at async link (node:internal/modules/esm/module_job:70:21) {
  code: 'ERR_UNKNOWN_MODULE_FORMAT'
}

It seems like since I've updated to Node.js v17.6.0 it's exclusively erroring with ERR_UNKNOWN_MODULE_FORMAT, but I was previously able to provoke a ERR_INVALID_MODULE_SPECIFIER by adding or removing file:/// for the file argument.

nodejs/node#42098 suggests that the code that throws ERR_UNKNOWN_MODULE_FORMAT is very new (less than 2 weeks old) and we're well into experimental territory here, but I just thought it was curious that this is only a problem on Windows.

Steps to reproduce the problem

Ended up just using the previous gist:

$ npx https://gist.github.com/brianjenkins94/d73578d7439427a59df28c1bc751e02d

Minimal reproduction

https://gist.github.com/brianjenkins94/d73578d7439427a59df28c1bc751e02d

Specifications

• ts-node version: v10.7.0
• node version: v17.6.0
• TypeScript version: v4.6.2

tsconfig.json

{
	"compilerOptions": {
		"allowJs": true,
		"esModuleInterop": true,
		"forceConsistentCasingInFileNames": false,
		"incremental": true,
		"isolatedModules": true,
		"jsx": "preserve",
		"lib": [
			"dom",
			"dom.iterable",
			"esnext"
		],
		"module": "ESNext",
		"moduleResolution": "Node",
		"noEmit": true,
		"resolveJsonModule": true,
		"skipLibCheck": true,
		"strict": false,
		"alwaysStrict": true,
		//"exactOptionalPropertyTypes": true,
		//"strictNullChecks": true,
		"strictBindCallApply": true,
		"strictFunctionTypes": true,
		//"strictPropertyInitialization": true,
		//"noImplicitAny": true,
		//"noImplicitThis": true,
		//"useUnknownInCatchVariables": true,
		"noImplicitOverride": true,
		"noImplicitReturns": true,
		"noPropertyAccessFromIndexSignature": true,
		"target": "ES2020"
	},
	"include": [
		"next-env.d.ts",
		"**/*.ts",
		"**/*.tsx"
	],
	"exclude": [
		"node_modules"
	]
}

• Operating system and version: Windows 10 21H2

Originally posted by @brianjenkins94 in #1663 (comment)

[cspotcode]

Does this work? Seems to be working on my end.

https://gist.github.com/cspotcode/d84d9bfe1b1d121075865c4ae5991850

[brianjenkins94]

Yes that appears to work... So what does that do differently?

Oh I see, it's a new feature, hot off the press.

[cspotcode]

Yeah, I didn't figure out what was going on with your reproduction cuz I figured I'd just try the new feature instead and see if it works.

We take care of spawning here: https://github.com/TypeStrong/ts-node/blob/main/src/child/spawn-child.ts
And we do some careful stuff to play nice with windows; possibly that's what's going on.

[cspotcode]

One thing I did notice:

path.join(baseDirectory, "..", "ts-node", "esm.mjs")

Assuming a certain relative path might break. Where, exactly, I'm not sure. But you can avoid that risk entirely with require.resolve('ts-node/esm.mjs')

Doesn't really matter if you're happy to use ts-node-esm instead.

[brianjenkins94]

Yeah, I'd be happy to get rid of my bootstrapper. The only thing it really does is force you to use a bash-like shell, which I can guard against in Node if it really matters. I'll give it a try tomorrow.

Thanks!

[cspotcode]

I am concurrently trying to create better documentation for this use-case. I've started working on an example project over here:
https://github.com/TypeStrong/ts-node-npx-example

However, it appears as though the shebang #!/usr/bin/env ts-node-esm requires ts-node-esm to already be on the PATH. If you don't have ts-node installed globally and you try to run npx in a non-project directory, you get an error that ts-node-esm cannot be found.

I might have to add a new function to our API to support users like you:

#!/usr/bin/env node
// npx.cjs

const path = require('path');
require('ts-node').execBin(`${ __dirname }/cli.ts`);

This execBin function can do the same work as the ts-node or ts-node-esm CLI, as if you'd written a shebang. But since you execute it with require(), your npx-installed code is sure to get the local dependency.

[cspotcode]

You should be able to use this trick in the meantime:
https://github.com/TypeStrong/ts-node-npx-example/blob/main/npx.cjs

[brianjenkins94]

Does this require a specific shebang on the .ts file? It seems to just be silently exiting for me.

Dumb question, I'll just try what you have:

#!/usr/bin/env -S npx ts-node --esm

---

Yeah mine is just exiting silently for me. I'll have to meticulously compare mine to yours.

Seems unrelated to ts-node on further inspection.

[brianjenkins94]

It works, we're good! 👍 👍 👍

Thanks!

[cspotcode]

Excellent!

Please do file issues and/or message me on Discord if you hit any other problems. I want shebangs and npx to work well. And it seems due to #1686 (comment) I still have some work to do.