Dependency of a .cjs build may not be loaded correctly

[eric-burel]

Verify canary release

• I verified that the issue exists in Next.js canary release

Provide environment information

$ /code/stateof/stateofjs-next/node_modules/.bin/next info
/bin/sh: 1: pnpm: not found

Operating System:
  Platform: linux
  Arch: x64
  Version: #33~20.04.1-Ubuntu SMP Mon Feb 7 14:25:10 UTC 2022
Binaries:
  Node: 14.18.2
  npm: 6.14.15
  Yarn: 1.22.5
  pnpm: N/A
Relevant packages:
  next: 12.1.1-canary.6
  react: 17.0.2
  react-dom: 17.0.2

Done in 1.15s.

What browser are you using? (if relevant)

No response

How are you deploying your application? (if relevant)

No response

Describe the Bug

I am building a library of components.

• Package "my-package-tsup" has shared (ESM, IIFE, CJS), client-only (ESM, IIFE) and server-only (CJS, ESM) exports
• "my-package-tsup/server" (CJS) depends on "my-package-tsup-dependency" (CJS and ESM).
• So, server entrypoints are using CJS, while I allow shared code to use either CJS or ESM, with an exports field like this:

{
  "name": "my-package-tsup",
  "version": "1.0.0",
  "description": "",
  "main": "./dist/shared/index.js",
  "type": "module",
  "files": [
    "dist/"
  ],
  "exports": {
    ".": {
      "node": "./dist/server/index.cjs",
      "import": "./dist/shared/index.js"
    },
    "./server": "./dist/server/index.cjs", => this is the one that seems problematic.
    "./client": "./dist/client/index.js"
  },

So, when importing "my-package-tsup/server", Next will correctly load the "dist/server/index.cjs" file. But when this CJS file require it's subdependency, using a require call, it will try to pick the .js ESM export instead of sticking to the .cjs CJS export.

Here is the package.json of the dependency:

{
  "name": "my-package-tsup-dependency",
  "version": "1.0.0",
  "description": "",
  "main": "./dist/shared/index.js",
  "type": "module",
  "files": [
    "dist/"
  ],
  "exports": {
    ".": {
      "node": "./dist/server/index.cjs", => this is the one that should be preferred but instead Next.js complains about trying to require an ESM module. It might be tricked by the "type":"module" and not respect the exports for dependencies, while it works ok for top-level packages
      "import": "./dist/shared/index.js"
    },
    "./server": "./dist/server/index.cjs",
    "./client": "./dist/client/index.js"
  },

To rephrase this:
If a CJS top level package requires one if it's dependency, the exports field of the dependency does not seem to be respected. The ESM export will be loaded, instead of the CJS export.

Expected Behavior

CommonJS dependencies should also use the CommonJS export of their own subdependencies.

To Reproduce

It's not the easiest to reproduce:

• I can reproduce consistenly with this library of package: https://github.com/VulcanJS/vulcan-npm/tree/feature/datatable

git clone 
git checkout feature/datatable
yarn && yarn run build && yarn run publish:local

It will rely on Yalc, so to test you'll need to set it up in a Next app.
Then import @vulcanjs/graphql/server in your Next app and you'll end up with the dreaded https://nextjs.org/docs/messages/import-esm-externals error

• I am trying to build a simpler repro here: https://github.com/VulcanJS/npm-the-right-way/tree/feature/try-dep

Clone, install, and open the "with-tsup" page http://localhost:3000/with-tsup

I seem to be able to have this message: Module not found: Package path . is not exported from package /code/npm-the-right-way/demo-next-app/node_modules/my-package-tsup-dependency (see exports field in /code/npm-the-right-way/demo-next-app/node_modules/my-package-tsup-dependency/package.json).

It's different from the error I have in the other package but could be related, as if "exports" were not understood correctly.

This issue might related: #34956

[eric-burel]

If I remove "type":"module" and rename the ESM export to ".mjs" it works better, so I suspect that Next interprets "type":"module" as being an ESM module, while I think it should specifically interpret only ".js" files as ESM in this scenario, but still treat ".cjs" extensions as CommonJS. This affects only external subdependencies, not direct dependency of the Next app.

Edit: it doesn't seem to work in all scenarios either though...

[DopamineDriven]

I've found a workaround that keeps dts in the defineConfig and also plays nicely with nextjs -- was having one hell of a problem since I'm using turbo repo to publish an external package called indexer for a project at work (to sync sanity/algolia datasets/indices (respectively) and provide more config and less coding for projects consuming it)

two things stood out to me, the first being the swcMinify flag

• swcMinify: true

this will make nextjs reject require being resolved dynamically no matter how you modify the next.config.js custom webpack...

Have to set swcMinify to false or leave it out entirely to avoid fs and dynamic require related errors

Nextjs needs a point of reference to handle consumption of the tsup-bundled package in the absence of a dts sourcemap especially if it isn't a browser-oriented package that can be handled with next-transpile-modules

fix: create a root index.d.ts file and reference the types index.d.ts file in the package bundled by tsup

declare the package as a module as follows in the root of your nextjs repo (or whatever client-rendering repo it happens to be)

• index.d.ts

/// <reference types="@takedajobs/indexer/dist/index" />

declare module "@takedajobs/indexer";

---

Inside of my indexer package, I'm augmenting the NodeJS namespace in the root as well, using a global.d.ts file to support the required environmental variables

namespace NodeJS {
  interface ProcessEnv {
    ALGOLIA_SEARCH_KEY: string;
    ALGOLIA_WRITE_KEY: string;
    ALGOLIA_INDEX: string;
    ALGOLIA_APP_ID: string;
    ALGOLIA_INDICES?: string;
    SANITY_DATASET: string;
    SANITY_PROJECT_ID: string;
    SANITY_API_TOKEN: string;
    SANITY_API_VERSION: string;
    SANITY_DATASET_TAG?: string;
    SANITY_REVALIDATE_SECRET?: string;
    SANITY_PREVIEW_SECRET?: string;
  }
}

Reinforced with a takeda.config.yaml file in the root of the package

this way the env variables can be extracted from the .env directly or streamed using js-yaml and a couple well placed buffers then passed into dotenv-expand where their values are exposed

if any of the required variables are missing it throws an error to the user of the package stating which env variable they're missing in their config

import path from "node:path";
import * as dotenv from "dotenv";

dotenv.config({ path: path.join(process.cwd(), ".env") });

import { expand } from "dotenv-expand";
import { YamlStreamer } from "../utils/yaml-streamer";

import type { DotenvExpandOutput } from "dotenv-expand";
import type { Indexer } from "../types/namespace";

export const defaultPath = () => "takeda.config.yaml" as const;

export const SecretObj = Array.of<NonNullable<DotenvExpandOutput["parsed"]>>();

export const configPath = (path?: `${string}`): string | "takeda.config.yaml" =>
  path ? path : defaultPath();

export const streamYamlFile = (path?: string) => YamlStreamer(
  configPath(path)
) as Indexer.Yaml.RootConfig;

export const YamlSecrets = (streamYamlFile: Indexer.Yaml.RootConfig) => {
  const expanded = expand({
    parsed: {
      ["ALGOLIA_INDICES"]:
        streamYamlFile.algoliaConfig?.alternativeIndices ?? "",
      ["ALGOLIA_APP_ID"]: streamYamlFile.algoliaConfig?.appId ?? "",
      ["ALGOLIA_SEARCH_KEY"]: streamYamlFile.algoliaConfig?.searchApiKey ?? "",
      ["ALGOLIA_WRITE_KEY"]: streamYamlFile.algoliaConfig?.writeApiKey ?? "",
      ["ALGOLIA_INDEX"]: streamYamlFile.algoliaConfig?.primaryIndex ?? "",
      ["SANITY_PROJECT_ID"]: streamYamlFile.sanityConfig?.projectId ?? "",
      ["SANITY_DATASET"]: streamYamlFile.sanityConfig?.dataset ?? "",
      ["SANITY_API_TOKEN"]: streamYamlFile.sanityConfig?.token ?? "",
      ["SANITY_REVALIDATE_SECRET"]:
        streamYamlFile.sanityConfig?.revalidateSecret ?? "",
      ["SANITY_DATASET_TAG"]: streamYamlFile.sanityConfig?.tag ?? "",
      ["SANITY_PREVIEW_SECRET"]:
        streamYamlFile.sanityConfig?.previewSecret ?? "",
      ["SANITY_API_VERSION"]: streamYamlFile.sanityConfig?.apiVersion ?? ""
    }
  });

  SecretObj.push(expanded.parsed!);
  return expanded.parsed!;
};

// consume augmented NodeJS.ProcessEnv interface defined in global.d.ts

type ENV = { [P in keyof NodeJS.ProcessEnv]?: NodeJS.ProcessEnv[P] };
interface ENVExtended extends ENV {}

type Config = { [P in keyof NodeJS.ProcessEnv]: NodeJS.ProcessEnv[P] };

interface ConfigExtended extends Config {}

// Loading process.env as ENV interface

const getConfig = (): ENVExtended => {
  return {
    ALGOLIA_APP_ID: process.env.ALGOLIA_APP_ID
      ? process.env.ALGOLIA_APP_ID
      : undefined,
    ALGOLIA_INDEX: process.env.ALGOLIA_INDEX
      ? process.env.ALGOLIA_INDEX
      : undefined,
    ALGOLIA_INDICES: process.env.ALGOLIA_INDICES
      ? process.env.ALGOLIA_INDICES
      : undefined,
    ALGOLIA_SEARCH_KEY: process.env.ALGOLIA_SEARCH_KEY
      ? process.env.ALGOLIA_SEARCH_KEY
      : undefined,
    ALGOLIA_WRITE_KEY: process.env.ALGOLIA_WRITE_KEY
      ? process.env.ALGOLIA_WRITE_KEY
      : undefined,
    SANITY_API_TOKEN: process.env.SANITY_API_TOKEN
      ? process.env.SANITY_API_TOKEN
      : undefined,
    SANITY_API_VERSION: process.env.SANITY_API_VERSION
      ? process.env.SANITY_API_VERSION
      : undefined,
    SANITY_DATASET: process.env.SANITY_DATASET
      ? process.env.SANITY_DATASET
      : undefined,
    SANITY_DATASET_TAG: process.env.SANITY_DATASET_TAG
      ? process.env.SANITY_DATASET_TAG
      : undefined,
    SANITY_PREVIEW_SECRET: process.env.SANITY_PREVIEW_SECRET
      ? process.env.SANITY_PREVIEW_SECRET
      : undefined,
    SANITY_PROJECT_ID: process.env.SANITY_PROJECT_ID
      ? process.env.SANITY_PROJECT_ID
      : undefined,
    SANITY_REVALIDATE_SECRET: process.env.SANITY_REVALIDATE_SECRET
      ? process.env.SANITY_REVALIDATE_SECRET
      : undefined
  };
};

const getSanitzedConfig = (config: ENVExtended): ConfigExtended => {
  for (const [key, value] of Object.entries(config)) {
    if (value === undefined) {
      throw new Error(`Missing key ${key} in .env`)
    }
  }
  return config as Config;
};

const config = getConfig();

export const {
  ALGOLIA_APP_ID,
  ALGOLIA_INDEX,
  ALGOLIA_SEARCH_KEY,
  ALGOLIA_WRITE_KEY,
  ALGOLIA_INDICES,
  SANITY_API_TOKEN,
  SANITY_API_VERSION,
  SANITY_DATASET,
  SANITY_DATASET_TAG,
  SANITY_PREVIEW_SECRET,
  SANITY_PROJECT_ID,
  SANITY_REVALIDATE_SECRET
} = getSanitzedConfig(config);

// yamlstream fallback method -- resolving how to make the two work in tandem 
// export const {
//   ALGOLIA_APP_ID = process.env.ALGOLIA_APP_ID,
//   ALGOLIA_INDEX = process.env.ALGOLIA_INDEX,
//   ALGOLIA_SEARCH_KEY = process.env.ALGOLIA_SEARCH_KEY,
//   ALGOLIA_WRITE_KEY = process.env.ALGOLIA_WRITE_KEY,
//   ALGOLIA_INDICES = process.env.ALGOLIA_INDICES,
//   SANITY_API_TOKEN = process.env.SANITY_API_TOKEN,
//   SANITY_API_VERSION = process.env.SANITY_API_VERSION,
//   SANITY_DATASET = process.env.SANITY_DATASET,
//   SANITY_DATASET_TAG = process.env.SANITY_DATASET_TAG,
//   SANITY_PREVIEW_SECRET = process.env.SANITY_PREVIEW_SECRET,
//   SANITY_PROJECT_ID = process.env.SANITY_PROJECT_ID,
//   SANITY_REVALIDATE_SECRET = process.env.SANITY_REVALIDATE_SECRET
// } = YamlSecrets(streamYamlFile);

Point being, you can make next play nicely and only use the cjs export from tsup + that declaration file triple-slash directive

Here's my tsup.config.ts

import { defineConfig } from "tsup";

const isProd = process.env.NODE_ENV === "production";

export default defineConfig({
  clean: true,
  dts: true,
  entry: ["src/index.ts"],
  format: ["cjs"],
  minify: isProd,
  tsconfig: "tsconfig.json",
  splitting: false,
  sourcemap: true
})

tsconfig.json

{
  "extends": "@takedajobs/tsconfig/node16.json",
  "include": ["**/*.ts", "tsup.config.ts", "global.d.ts", "takeda.config.yaml", "."],
  "exclude": ["node_modules"]
}

the tsconfig it is extending

{
  "$schema": "https://json.schemastore.org/tsconfig",
  "compilerOptions": {
    "composite": false,
    "declaration": true,
    "declarationMap": true,
    "inlineSources": false,
    "isolatedModules": true,
    "moduleResolution": "Node",
    "noFallthroughCasesInSwitch": true,
    "noStrictGenericChecks": false,
    "noUnusedLocals": false,
    "noUnusedParameters": false,
    "pretty": true,
    "preserveWatchOutput": true,
    "noImplicitThis": true,
    "strict": true,
    "alwaysStrict": true,
    "lib": ["ESNext", "DOM", "DOM.Iterable"],
    "module": "CommonJS",
    "target": "ES2021",
    "outDir": "dist",
    "esModuleInterop": true,
    "skipLibCheck": true,
    "forceConsistentCasingInFileNames": true
  }
}

package.json

{
  "name": "@takedajobs/indexer",
  "version": "1.0.0",
  "main": "./dist/index.js",
  "module": "./dist/index.mjs",
  "types": "./dist/index.d.ts",
  "sideEffects": false,
  "files": [
    "dist/**"
  ],
  "scripts": {
    "prebuild": "yarn regenerate",
    "build": "tsup",
    "predev": "yarn regenerate",
    "dev": "tsup --watch",
    "lint": "TIMING=1 eslint ./src/**/*.ts* --fix",
    "clean": "rm -rf .turbo && rm -rf node_modules && rm -rf dist",
    "regenerate": "rm -rf dist"
  },
  "devDependencies": {
    "@algolia/client-common": "^4.14.2",
    "@takedajobs/tsconfig":"*",
    "@types/js-yaml": "^4.0.5",
    "@types/node": "^18.8.2",
    "dotenv": "^16.0.3",
    "dotenv-cli": "^6.0.0",
    "dotenv-expand": "^9.0.0",
    "eslint": "^8.24.0",
    "eslint-config-custom": "*",
    "ts-node": "^10.9.1",
    "tslib": "^2.4.0",
    "tsup": "^6.2.3",
    "typescript": "^4.9.1-beta"
  },
  "publishConfig": {
    "access": "public"
  },
  "dependencies": {
    "@sanity/client": "^3.4.1",
    "algoliasearch": "^4.14.2",
    "chalk": "^5.1.0",
    "js-yaml": "^4.1.0"
  }
}

Lastly, the logs of a local build I just ran to show that the augmented .env values are output in nextjs with no failures when using this approach

dopaminedriven@LAPTOP-2IH011V4:~/takeda/HCMS-TakedaJobs$ yarn turbo run build
yarn run v1.22.19
$ /home/dopaminedriven/takeda/HCMS-TakedaJobs/node_modules/.bin/turbo run build
• Packages in scope: @takedajobs/example, @takedajobs/indexer, @takedajobs/studio, @takedajobs/tsconfig, @takedajobs/ui, @takedajobs/utils, @takedajobs/vercel, eslint-config-custom
• Running build in 8 packages
 INFO  • Remote caching enabled
@takedajobs/indexer:build: cache hit, replaying output f02ab328073e34a1
@takedajobs/indexer:build: warning package.json: No license field
@takedajobs/indexer:build: $ yarn regenerate
@takedajobs/indexer:build: warning package.json: No license field
@takedajobs/indexer:build: $ rm -rf dist
@takedajobs/indexer:build: $ tsup
@takedajobs/indexer:build: CLI Building entry: src/index.ts
@takedajobs/indexer:build: CLI Using tsconfig: tsconfig.json
@takedajobs/indexer:build: CLI tsup v6.2.3
@takedajobs/indexer:build: CLI Using tsup config: /home/dopaminedriven/takeda/HCMS-TakedaJobs/packages/indexer/tsup.config.ts
@takedajobs/indexer:build: CLI Target: node14
@takedajobs/indexer:build: CLI Cleaning output folder
@takedajobs/indexer:build: CJS Build start
@takedajobs/indexer:build: CJS dist/index.js     12.25 KB
@takedajobs/indexer:build: CJS dist/index.js.map 34.08 KB
@takedajobs/indexer:build: CJS ⚡️ Build success in 33ms
@takedajobs/indexer:build: DTS Build start
@takedajobs/indexer:build: DTS ⚡️ Build success in 2618ms
@takedajobs/indexer:build: DTS dist/index.d.ts 8.20 KB
@takedajobs/studio:build: cache hit, replaying output 84ecf07ece8ad14f
@takedajobs/studio:build: $ sanity build dist -y
@takedajobs/studio:build: - Clearing output folder
@takedajobs/studio:build: ✔ Clearing output folder (6ms)
@takedajobs/studio:build: - Building Sanity
@takedajobs/studio:build: ✔ Building Sanity (29529ms)
@takedajobs/studio:build: - Building index document
@takedajobs/studio:build: ✔ Building index document (63ms)
@takedajobs/studio:build: - Minifying JavaScript bundles
@takedajobs/studio:build: ✔ Minifying JavaScript bundles (47120ms)
@takedajobs/vercel:build: cache miss, executing 533452ca23cd6578
@takedajobs/example:build: cache miss, executing fe1f1713b3acbae1
@takedajobs/vercel:build: $ next build
@takedajobs/example:build: $ next build
@takedajobs/vercel:build: info  - Loaded env from /home/dopaminedriven/takeda/HCMS-TakedaJobs/apps/vercel/.env.local
@takedajobs/vercel:build: info  - Loaded env from /home/dopaminedriven/takeda/HCMS-TakedaJobs/apps/vercel/.env
@takedajobs/example:build: info  - Loaded env from /home/dopaminedriven/takeda/HCMS-TakedaJobs/apps/example/.env.local
@takedajobs/example:build: info  - Loaded env from /home/dopaminedriven/takeda/HCMS-TakedaJobs/apps/example/.env
@takedajobs/example:build: warn  - You have enabled experimental feature (fontLoaders) in next.config.js.
@takedajobs/example:build: warn  - Experimental features are not covered by semver, and may cause unexpected or broken application behavior. Use at your own risk.
@takedajobs/example:build: 
@takedajobs/vercel:build: info  - Linting and checking validity of types...
@takedajobs/example:build: info  - Linting and checking validity of types...
@takedajobs/example:build: info  - Creating an optimized production build...
@takedajobs/example:build: info  - Using tsconfig file: ./tsconfig.json
@takedajobs/example:build: info  - Compiled successfully
@takedajobs/example:build: info  - Collecting page data...
@takedajobs/example:build: 60f0a979f021a303eb3aa4bdddb84a3e764e785ebf2de79255f3cb0aea9fa679c442ae00c4afded7e805bedf210f3e036184d98859b5b46deb3e8ecc1a5c83dc698c91c94404b4ce2a2f64546e4734564821f3aa97997220b48d
@takedajobs/example:build: 60f0a979f021a303eb3aa4bdddb84a3e764e785ebf2de79255f3cb0aea9fa679c442ae00c4afded7e805bedf210f3e036184d98859b5b46deb3e8ecc1a5c83dc698c91c94404b4ce2a2f64546e4734564821f3aa97997220b48d
@takedajobs/example:build: 60f0a979f021a303eb3aa4bdddb84a3e764e785ebf2de79255f3cb0aea9fa679c442ae00c4afded7e805bedf210f3e036184d98859b5b46deb3e8ecc1a5c83dc698c91c94404b4ce2a2f64546e4734564821f3aa97997220b48d
@takedajobs/example:build: info  - Generating static pages (0/3)
@takedajobs/example:build: 60f0a979f021a303eb3aa4bdddb84a3e764e785ebf2de79255f3cb0aea9fa679c442ae00c4afded7e805bedf210f3e036184d98859b5b46deb3e8ecc1a5c83dc698c91c94404b4ce2a2f64546e4734564821f3aa97997220b48d
@takedajobs/example:build: 60f0a979f021a303eb3aa4bdddb84a3e764e785ebf2de79255f3cb0aea9fa679c442ae00c4afded7e805bedf210f3e036184d98859b5b46deb3e8ecc1a5c83dc698c91c94404b4ce2a2f64546e4734564821f3aa97997220b48d
@takedajobs/example:build: 60f0a979f021a303eb3aa4bdddb84a3e764e785ebf2de79255f3cb0aea9fa679c442ae00c4afded7e805bedf210f3e036184d98859b5b46deb3e8ecc1a5c83dc698c91c94404b4ce2a2f64546e4734564821f3aa97997220b48d
@takedajobs/example:build: info  - Generating static pages (3/3)
@takedajobs/example:build: info  - Finalizing page optimization...
@takedajobs/example:build: 
@takedajobs/example:build: Route (pages)                              Size     First Load JS
@takedajobs/example:build: ┌ ○ /                                      2.48 kB         116 kB
@takedajobs/example:build: ├   └ css/9624720adf640a1b.css             1.22 kB
@takedajobs/example:build: ├   /_app                                  0 B             113 kB
@takedajobs/example:build: └ ○ /404                                   194 B           113 kB
@takedajobs/example:build: + First Load JS shared by all              117 kB
@takedajobs/example:build:   ├ chunks/framework-7dc8a65f4a0cda33.js   45.2 kB
@takedajobs/example:build:   ├ chunks/main-9800b5260862a44b.js        31.1 kB
@takedajobs/example:build:   ├ chunks/pages/_app-7f3d0cbda28fef3c.js  36.1 kB
@takedajobs/example:build:   ├ chunks/webpack-5752944655d749a0.js     840 B
@takedajobs/example:build:   └ css/e766f52bde8757b7.css               3.32 kB
@takedajobs/example:build: 
@takedajobs/example:build: ○  (Static)  automatically rendered as static HTML (uses no initial props)
@takedajobs/example:build: 

The string being logged over and over again

@takedajobs/example:build: 60f0a979f021a303eb3aa4bdddb84a3e764e785ebf2de79255f3cb0aea9fa679c442ae00c4afded7e805bedf210f3e036184d98859b5b46deb3e8ecc1a5c83dc698c91c94404b4ce2a2f64546e4734564821f3aa97997220b48d

corresponds to this code in _app.tsx of the nextjs example app

import "../styles/index.css";
import type { AppProps } from "next/app";
import cn from "clsx";
import { inter } from "../fonts";
import { SANITY_PREVIEW_SECRET, SANITY_API_TOKEN } from "@takedajobs/indexer";

console.log(SANITY_API_TOKEN);

function MyApp({ pageProps, Component }: AppProps) {
  return (
    <div
      className={cn(
        inter.className,
        "min-w-screen mx-auto min-h-screen w-fit antialiased"
      )}>
      <Component {...pageProps} />
    </div>
  );
}

export default MyApp;

hopefully this helps, can't believe I just spent 2-3 hours tweaking this to work lol

[ecyrbe]

I confirm this issue is happening for any project using cjs only. Here is another reproductible issue :
TanStack/query#4346

[nextjs-bot]

This issue has been automatically marked as stale due to two years of inactivity. It will be closed in 7 days unless there’s further input. If you believe this issue is still relevant, please leave a comment or provide updated details. Thank you.

[nextjs-bot]

This issue has been automatically closed due to two years of inactivity. If you’re still experiencing a similar problem or have additional details to share, please open a new issue following our current issue template. Your updated report helps us investigate and address concerns more efficiently. Thank you for your understanding!

[github-actions]

This closed issue has been automatically locked because it had no new activity for 2 weeks. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.