Version 3

[cyrilwanner]

Background

This project started out as a small package. Over time, more and more features were added and it now provides many different things and is used by many developers. But I'm not 100% happy with the current version for the following reasons:

• it depends on many different webpack loaders which has led to some workarounds and many features cannot be combined because the loaders don't work well with each other. See Inlining resized images #84, webp srcset #91 for examples.
• those loaders also depend on other packages which have also introduced errors/bugs where we don't have the possibility to fix them directly (as in svg?sprite Cannot find module svg-baker-runtime/symbol.js #75).
• while this package grew, the installation and configuration became more and more complex and I think there is some space for improvement here.

Idea

To make this package robust for the future and easy to maintain, I think the only way is to fix those issues. And this probably requires a rewrite of the whole package.

It can be broken down into multiple parts:

Webpack loaders

This is the most important part of this package and is currently out of our control. To support use-cases such as mentioned in #84 or #91 and so we are able to provide a stable plugin, I think the only way is to write an own webpack loader which does the image optimizations. That would have the following advantages:

• all features work together (e.g. you can resize a jpg, convert it to webp and generate a lqip all at the same time, which is not possible right now)
• we have more control over the libraries used for image optimization (that part would of course still be a 3rd party library)
• it would be published as a separate loader and could also be used outside of a next.js project

Next.js plugin

This plugin would configure the independent webpack loader for easy use with next.js. It should also support the upcoming next.js plugin system for an easy installation.

Support run-time optimization

One of the most requested features was to be able to optimize images during run-time (e.g. from an API) and not only local images during build-time (see #104, #77, #74). With the API routes introduced in next.js 9, I think it should be possible to provide an easy-to-use service for that. It obviously can't relay on webpack, but the code used in the webpack loader should be reusable.
That would have the following advantages:

• it is possible to optimize images returned from an API/3rd party service
• they would have exactly the same optimization options and query strings so you can use them exactly the same as a local image

But this feature would need to be defined more detailed (e.g. it would need a store, like S3, so images are not optimized on every request, and this API would be best located behind a CDN, etc.).

Provide image components

Creating an image component is not easy, especially if you want to support all features of this plugin (webp fallback, lqip, srcset, ...). And you still have to think about what happens when JS is disabled (while lazy loading), how it has to be optimized for screen readers and search engines and so on. So if you want your images to be working well for everyone, it is not just <img src={require('./image.jpg')} />, there is a lot more work to do and you don't want to do this again in every project.

So I'm thinking about providing an image component that will work well with this plugin and already does most of the things for you (non-JS fallback, a11y, lazy loading with fallbacks, ...).

It should support the following things:

• webp with fallbacks for old browsers (see WebP auto fallback #111)
• lazyloading using either a low quality image placeholder (Blur Up), Image Outlines or Dominant Colors

Additionally, it would be nice to provide a babel plugin which makes it even easier to use this component.
So instead of this:

<Img
  src={require('./image.jpg')}
  sizes={{
    100: require('./image.jpg?resize&size=100'),
    300: require('./image.jpg?resize&size=300'),
    600: require('./image.jpg?resize&size=600')
  }}
  lqip={require('./image.jpg?lqip')}
/>

You would only use it like this and the babel plugin will transform it to the way more complex way above:

<Img src={require('./image.jpg')} sizes={[100, 300, 600]} lqip />

Feedback wanted

I am of course open to any feedback on any part of this idea. Also, if you have ideas of new features, now would be the time as they could be included from the beginning of the new version.

I am not quite sure on which image optimization library this should be based on. Right now, it is mainly imagemin but I am not really happy with it since it is not easy to install, especially in some CI/CD environments since it requires specific binaries to be installed on the system.
So I was thinking about using sharp as the main optimization library as it is a lot easier to install, has a great node API and is still quite fast compared to other node libraries.

Feedback on this would be very appreciated, especially:

• has someone made experiences with both libraries? (regarding speed, simplicity, ...)
• would you switch to another library?
• if the decision falls for sharp, should imagemin (or others) also be additionally supported?

Is there an ETA?

No, development has started but is in the early stages. The features mentioned above are quite complex and require a lot of work, so it will probably take some weeks. But I will update this issue when progress is made and when a canary version is available.

[AleVul]

Could i suggest to use wasm for the optimization part. Squoosh uses wasm to compress images, for example this lib

[cyrilwanner]

That's a good idea @AleVul, thanks! That would solve the problem with native binaries needed. I'll definitely check this option out 👍

[akd-io]

Sounds awesome @cyrilwanner, are you still pursuing this?

I think this project is important and would love to see it flourish. I have no experience with Webpack stuff, but let me know if you need help with any laborious tasks or more general JS/TS parts of the library.

[merelinguist]

Eagerly awaiting this! 😄

[xndyz]

Remote image optimization would be amazing. 🎉

[cyrilwanner]

I'm sorry for the long silence on this issue. I just didn't have much time to work on this over the last few months, unfortunately.
But yes, all this is still planned and will come.
Last week, I started to work on this issue again so I'll hopefully be able to share some progress soon.

Currently, I'm working on compiling the different image optimization tools into WebAssembly so we are able to make the whole installation (especially in a CI/CD environment) a lot easier and more consistent. Squoosh, unfortunately, doesn't provide their compiled WASM binaries as packages, but I'm slowly getting there.
That's probably the biggest and most time-consuming part. So once that is done, the other features are way easier to implement.

[akd-io]

Super cool Cyril, best of luck! 💪

[jasonsilberman]

I saw your message on the Next.js discussions board and posted a message in response to another user's questions with how I did something like this. I'd be happy to help out or give a few pointers with how I implemented something like this!

[cyrilwanner]

Hi @jasonsilberman and thank you very much!
I'm currently still on the foundation for all those new features, but I'll come back to you when it is ready and the remote image optimization can get implemented as I saw in your comment that you already did some work in that area.

But I already have a few initial questions, if you would like to give some insights.
It looks like the images get resized during upload and not on-the-fly when they get requested, is that correct? Also, did you deploy this into a cloud/serverless environment? If so, did you encounter any issues with sharp? Or any other things we should already consider or think about when planning/implementing this feature?
Thanks for your help!

[tilman]

Hey, really looking forward to the new Version! In the meantime I had the need for an Image Component by myself so I created one and shared it here:
https://github.com/tilman/next-optimized-image-component

It's still WIP but supports:

• lqip image placeholder, blurred with SVG filter for best compatibility
• blur radius adapts automatically based on the image size. Because we can't determine displayed image size in SSR, it is animated to this radius from a default radius of 40px
• If the IntersectionObserver API is available, it only starts loading the image if it is near the viewport
• use all height/width combinations for the Img component and the component will take care about placing the lqip exactly over the image
• use webp as picture source by default with fallback for older browsers
• also supports a no javascript fallback

Since you wrote, you want to provide an Image Components as well, I thought it's maybe interesting for you.

[truongsinh]

Feedback on provided component:

Instead of using babel transform, we can use gatsby-image strategy. Something like this:

class Lquip {
  imgSrc: string;
  blur: number;
}
class LquipColor {
  backgroundColor: string;
}
class Trace {
  imgSrc: string;
}
interface ResponsivePictureElement extends DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement> {
  resources: {
    sources: DetailedHTMLProps<SourceHTMLAttributes<HTMLSourceElement>, HTMLSourceElement>[]
    fallbackSrc: string;
    placeholder?: Lquip | LquipColor | Trace;
    intrinsicWidth: number;
    intrinsicHeight: number
  }
}

export function Picture(props: ResponsivePictureElement) {
  const { resources: { sources, fallbackSrc, placeholder, intrinsicWidth, intrinsicHeight } } = props;
  let { style } = props
  if (placeholder instanceof Lquip) {
    // TODO can `filter` be applied to `backgroundImage` ?
    style = { ...style, backgroundImage: `url('${placeholder.imgSrc}')`, filter: `blur(${placeholder.blur}px)` }
  } else if (placeholder instanceof LquipColor) {
    style = { ...style, backgroundColor: placeholder.backgroundColor }
  } else if (placeholder instanceof Lquip) {
    style = { ...style, backgroundImage: `url('${placeholder.imgSrc}')` }
  }
  style = { ...style, height: intrinsicHeight, width: intrinsicWidth }
  return <picture {...props} style={style}>
    {sources.map((r) => <source
      srcSet={r.srcSet}
      media={r.media}
      sizes={r.sizes}
      type={r.type}
      {...r}
    />)}
    <img src={fallbackSrc} />
  </picture>
}

usage

<Picture resources={require('./myimage.png')} />
<Picture resources={require('./myimage.png?trace&resize&size[]=300&webp')} />

The 2 above can be equivalent with preset in the config.

WDYT?

[jasonsilberman]

Hi @jasonsilberman and thank you very much!
I'm currently still on the foundation for all those new features, but I'll come back to you when it is ready and the remote image optimization can get implemented as I saw in your comment that you already did some work in that area.

But I already have a few initial questions, if you would like to give some insights.
It looks like the images get resized during upload and not on-the-fly when they get requested, is that correct? Also, did you deploy this into a cloud/serverless environment? If so, did you encounter any issues with sharp? Or any other things we should already consider or think about when planning/implementing this feature?
Thanks for your help!

You are right, I am processing images on upload and not on the fly. I did this because we host the images on Google Storage (with a CDN in front) and so we could not easily do resizing on the fly. Additionally, this is for a static site and we know all of the images dimensions ahead of time, so we could pre-cut all of the exact images we needed.

We have deployed our image pipeline to a serverless environment, Vercel. I have not experienced any issues with using Sharp on Vercel's platform as of yet. The one thing to be consider is that you can only write to a /tmp folder on Vercel, and so you could not save images to disk. We use the readStream and writeStream APIs so this was not an issue for us.

Hope this helps.

---

Some other thoughts on the plugin in general:

• I think it would be important that you can download/upload images to S3 or other cloud providers. As this would allow images to be served over CDN, or perhaps use already uploaded user-content as your image.
• I appreciate that you are making sure to take into considerations the accessibility, performance, and SEO implications of this plugin. I think this is also important for today's websites.
• Lastly, while Gatsby's tools only work at build-time because of the nature of Gatsby, it would be great if we could find a way to make this work on both static and dynamic sites. I agree that it seems API Routes (for serverless deployments) or some middleware that could be used with custom servers would be ideal!

Anyway, I'm excited about the development of this plugin!

[brandonpittman]

I surprised Vercel hasn't built an image component yet. They have ones for head and link. 🤔