Include async chunks in the entrypoints file

[stof]

To preload or eagerly load async chunks on some pages, we would need to know the list of files necessary for that chunk (and to assign a name explicitly for the chunk to know how to get the right one, but that part is already possible with webpack and happens in the source code when importing the async chunk). AFAICT, including them in the entrypoints.json file should be enough to solve this. But I'm not sure assets-webpack-plugin supports that.

[stof]

@weaverryan do you think this is possible.

[Lyrkan]

@stof What would be the difference with /* webpackPreload: true */ and /* webpackPrefetch: true */?

[stof]

These won't work in the context of Symfony + Encore, as we don't render the HTML with webpack and so it cannot output a <link rel="preload"> tag.

But my use case is indeed to be able to add such <link> tag.
Thus, in my case, I would like to preload some imported modules only on some of the pages, not everytime the main bundle is loaded (as I know some of my dynamic imports will be used only for specific features, for which I can know which page are very likely to use them and so should preload the chunk). So I would not want Encore to force bundling this knowledge into the entrypoint data itself. Having files needed for the different dynamic chunks (similar to how files are listed for entrypoints) would allow to render these.

[Lyrkan]

These won't work in the context of Symfony + Encore, as we don't render the HTML with webpack and so it cannot output a tag.

I'm 90% sure it would also work in a Symfony context. Having these inline directives when doing a dynamic import will make Webpack add <link> tags at runtime (I know for sure that it works for JS since I use it on some projects).

So, as long as you have the right public path you should end-up with your JS/CSS files preloaded as soon as the main JS file is executed.

I'm not sure it would cover your whole use case though since prefetching chunks only on some pages would require you to have the dynamic import in files that are specific to those pages. I'll look into it, but I don't think exposing async chunks in the entrypoints.json will be an easy thing to do.

[Lyrkan]

Update: it seems that it actually doesn't work if you import(/* webpackPrefetch: true */ 'something.css') because Webpack will add a <link> tag for the generated (and almost empty) .js chunk instead of the .css file.

We'd probably have the same issue if we tried to add async chunks info to the entrypoints.json file.

[Lyrkan]

@stof I'm wondering... would disabling the entrypoints option of the assets plugin fit your need?

const Encore = require('@symfony/webpack-encore');
const AssetsWebpackPlugin = require('assets-webpack-plugin');

// ...

const config = Encore.getWebpackConfig();
for (const plugin of config.plugins) {
  if (plugin instanceof AssetsWebpackPlugin) {
    plugin.options.entrypoints = false;
  }
}

module.exports = config;

We don't have a method to directly configure the assets-webpack-plugin but that could be added quite easily.

[stof]

@Lyrkan this does not help. This includes all chunks in the file, but does not group them together by entrypoints (initial or async ones). What I need is to include the async entrypoints (not sure what the webpack terminology is here)

[stof]

I'm not sure it would cover your whole use case though since prefetching chunks only on some pages would require you to have the dynamic import in files that are specific to those pages. I'll look into it, but I don't think exposing async chunks in the entrypoints.json will be an easy thing to do.

My dynamic import is inside some widget code, which gets used only if the widget is in the DOM for my page. With hundreds of different pages in my app (not a SPA at all), I don't have a separate entrypoint for each page (that would be insane regarding caching for my users if any navigation gets a cold cache for JS. I only have 2 entrypoints, one for the admin section and one for normal pages). My use case is that I have an emoji-picker widget, which depends on the the emojione library (to have the list of available emojis), which is quite big. As only some pages will use this emoji-picker (and that I need my async chunk only if they interact with the picker), I don't want to bundle it on all pages (hence the async chunk for code splitting). But I know that my homepage has this emoji-picker widget, and that there is a good chance that users will use it there. So I would like to preload this async chunk only for the homepage, but not for other pages using the same JS entrypoint. This is why I'm asking to expose that info to WebpackEncoreBundle and not asking webpack to inject a <link> tag on load.

[Lyrkan]

This includes all chunks in the file, but does not group them together by entrypoints (initial or async ones)

Yeah... I didn't notice that in the quick test I did.

Thank you for the explanation about your use case, I think that we could make it work if we had access to stats.namedChunkGroups in the function that we use to change the content of the entrypoints.json just before it is created (if the async chunks are named... but this feature wouldn't make sense if they aren't).

For instance with two files main.js and admin.js that both asynchronously import emoji-picker.js (which statically imports emojione) we get:

• in stats.entrypoints (which is used by the assets-webpack-plugin when entrypoints is true):

{
  main: {
    chunks: [Array],
    assets: [Array],
    children: [Object: null prototype] {},
    childAssets: [Object: null prototype] {},
    isOverSizeLimit: undefined
  },
  admin: {
    chunks: [Array],
    assets: [Array],
    children: [Object: null prototype] {},
    childAssets: [Object: null prototype] {},
    isOverSizeLimit: undefined
  }
}

• in stats.namedChunkGroups:

{
  main: {
    chunks: [Array],
    assets: [Array],
    children: [Object: null prototype] {},
    childAssets: [Object: null prototype] {},
    isOverSizeLimit: undefined
  },
  admin: {
    chunks: [Array],
    assets: [Array],
    children: [Object: null prototype] {},
    childAssets: [Object: null prototype] {},
    isOverSizeLimit: undefined
  },
  'emoji-picker': {
    chunks: [Array],
    assets: [Array],
    children: [Object: null prototype] {},
    childAssets: [Object: null prototype] {},
    isOverSizeLimit: undefined
  }
}

By patching assets-webpack-plugin to use stats.namedChunkGroups instead of stats.entrypoints here and here we actually get the following content for this example:

{
  "entrypoints": {
    "main": {
      "js": [
        "/runtime.9533abfb.js",
        "/main.e27d46c2.js"
      ]
    },
    "admin": {
      "js": [
        "/runtime.9533abfb.js",
        "/admin.0803baac.js"
      ]
    },
    "emoji-picker": {
      "js": [
        "/vendors~emoji-picker.da867b41.js",
        "/emoji-picker.11d234d3.js"
      ]
    }
  }
}

Now the problem is that we don't have access to the stats object where we need to (that's something the plugin lacks in my opinion)...

[stof]

Relying on stats.namedChunkGroups and getting this output would be exactly what I need.

[Lyrkan]

@stof I quickly hacked something (a plugin that modifies the content of the entrypoints.json file after it's created), not sure if that's the best approach but would you be able to give it a shot?

$ yarn add git://github.com/lyrkan/webpack-encore.git#add-named-group-entrypoints

I'm still thinking about whether or not these should be added under the entrypoints key since they are not really entrypoints... but it would make it easier to use them with the bundle and our users probably wouldn't care about the terminology...

[abhinav-sendinblue]

If each entry point acts as a separate app and each one can have there own nested routes. How to split code in that case? Using React Lazy or React Loadable I am able to split the entry bundle but all the files still loaded by corresponding entry chunk.

[Lyrkan]

@abhinav-sendinblue I'm not sure to fully understand your comment but if two entrypoints async load the same thing you should get a common chunk. This chunk is then only loaded when needed (either from your code or from a vendor).

Anyway, I don't think this is really related to this issue since exposing more info in the entrypoints.json probably wouldn't help you.

[abhinav-sendinblue]

Let me share an example to explain my problem:-

I have 3 entry points:-

.addEntry('one', './assets/one/app.js')
.addEntry('two', './assets/two/app.js')
.addEntry('three', './assets/three/app.js')

Now all these entry points are separate react apps and they have their owned nested route like:-

<Route exact path="/one/1" component={Home} />
<Route exact path="/one/2" component={otherComponent} />

Same goes for other two entry points:-

<Route exact path="/two/1" component={Home} />
<Route exact path="/two/2" component={otherComponent} />

Now lets says for entry one when I hit

/one/1

or

/one/2

for both there is only one bundle one.js is being loaded.

I wanted to further split my chunks as per my nested routes. By using dynamic imports and React Loadable I am able to split chunks:-

one.js 
0.js
1.js

but these are being loaded together. All three no matter you hit /one/1 or /one/2.