8306707: Support pluggable image loading via javax.imageio

[mstr2]

This PR extends the image loading capabilities of JavaFX by plugging into the javax.imageio API, allowing applications to use third-party libraries like TwelveMonkeys to load a wide variety of image formats.

The built-in JavaFX image loaders continue to be used for all supported image formats. If an image can't be loaded using one of the built-in image loaders, JavaFX will probe the ImageIO registry for image readers that can load the image instead.

This PR also adds support for variable-density images like SVG. Variable-density images will be rasterized using the screen's pixel density, ensuring they are rendered at a higher resolution on high-DPI screens.

The proposed feature works transparently across all JavaFX image APIs, including images referenced by stylesheets, and images contained in data-URIs.

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issue

• JDK-8306707: Support pluggable image loading via javax.imageio (Enhancement - P4)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jfx.git pull/1093/head:pull/1093
$ git checkout pull/1093

Update a local copy of the PR:
$ git checkout pull/1093
$ git pull https://git.openjdk.org/jfx.git pull/1093/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 1093

View PR using the GUI difftool:
$ git pr show -t 1093

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jfx/pull/1093.diff

[bridgekeeper]

👋 Welcome back mstrauss! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[andy-goryachev-oracle]

[objection]

Is it possible to make it as an independent library, or does it require changes in the fx core to work?

The two issues I have with this proposal are:

1. it will require an automatic dependency on java.desktop
2. it's a half-measure, the actual fix would be to actually implement this functionality in the FX core

But as a library, the decision to include javax.desktop is trivial for an application.

What do you think?

[mstr2]

Is it possible to make it as an independent library, or does it require changes in the fx core to work?

It does require some changes to make variable-density images like SVG work correctly, and it requires adding a new ImageLoader, which is not API and thus cannot be done by a third-party library.

The two issues I have with this proposal are:

1. it will require an automatic dependency on java.desktop

javafx.graphics already requires java.desktop, no change here.

2. it's a half-measure, the actual fix would be to actually implement this functionality in the FX core

Well, it enables application developers to use any existing Image I/O plugin out there (of which there are many). Adding a random new image format directly to JavaFX every now and then feels more like a half-measure to me, as you'll always end up missing just that exotic format that someone so desperately needs.

Short of actually creating new API, adding more image formats directly to JavaFX doesn't solve the extensibility problem. And even if we did create new API in JavaFX, it bears the question why we would need to have two different imaging APIs in the Java world, one for which a large number of plugins already exist (javax.imageio), and one for which this is not the case. It's a tough sell for plugin authors to recreate all of those plugins for the very narrow use case of supporting a random JavaFX application out there.

[swpalmer]

On Apr 24, 2023, at 7:34 PM, mstr2 ***@***.***> wrote:

…

Short of actually creating new API, adding more image formats directly to JavaFX doesn't solve the extensibility problem. And even if we did create new API in JavaFX, it bears the question why we would need to have two different imaging APIs in the Java world, one for which a large number of plugins already exist (javax.imageio), and one for which this is not the case. It's a tough sell for plugin authors to recreate all of those plugins for the very narrow use case of supporting a random JavaFX application out there.

I have always thought that image loading in Java is a mess that needs a significant refactoring. ImageIO should be the only way images are loaded or saved (with the possible exception of splash screen support). It makes little sense to maintain multiple image loaders for the same formats. Image loading should not depend on java.desktop as image loading and saving is not exclusive to desktop applications. Both AWT and JavaFx image losing and saving should interface with ImageIO through primitive arrays, ByteBuffers, etc. such that specific java.desktop or JavaFx classes such as BufferedImage or WriteableImage are not required by the module that contains the ImageIO implementation. I realize this is out of scope for this proposal, but it would be great if things moved in that direction and the JavaFx dependencies on java.desktop can be reduced rather than become more ingrained. The fact that JavaFx and Swing are tied together when they should be independent is somewhat irritating when creating a reduced JRE for a bundled application. Just my 2c. Scott

[mstr2]

I realize this is out of scope for this proposal, but it would be great if things moved in that direction and the JavaFx dependencies on java.desktop can be reduced rather than become more ingrained. The fact that JavaFx and Swing are tied together when they should be independent is somewhat irritating when creating a reduced JRE for a bundled application. Just my 2c. Scott

I don't expect the modularization of Java to be significantly changed in any foreseeable future. But if it did, there would be nothing stopping us from changing the proposed implementation of this feature.

Postponing JavaFX image loading extensibility until the day comes when the JDK develops an Image I/O abstraction layer just means that JavaFX will not have that capability any time soon, maybe ever.

[yososs]

The current JavaFX IIO extension is
ImageStorage.getInstance().addImageLoaderFactory()
is possible.
Therefore, the changes to JavaFX IIO extension points are:
I think it would be better to change the module (module-info) of the ImageStorage class to public.

I think it would be better to focus on Vector Image support.

[mstr2]

The current JavaFX IIO extension is ImageStorage.getInstance().addImageLoaderFactory() is possible. Therefore, the changes to JavaFX IIO extension points are: I think it would be better to change the module (module-info) of the ImageStorage class to public.

If we did that, we'd have two image loading APIs in the Java world, and lacking suitable plugins, still no support for most other image formats. On the other hand, Image I/O plugins are readily available.

I think it would be better to focus on Vector Image support.

This PR also adds support for vector images. TwelveMonkey's SVG plugin works out of the box, as you'd expect it.

[yososs]

Image scaling is now also provided by ImageView.
For Vector Image, I thought of a way to use requestWidth, requestHeight in javafx Image class constructor.

public Image(String url,
  double requestedWidth,
  double requestedHeight,
  boolean preserveRatio,
  boolean smooth,
  boolean backgroundLoading)

Monitoring the size of the ImageView and calling setImage would probably work with the least amount of memory required.

[mstr2]

Image scaling is now also provided by ImageView.

Image rescaling is only meaningful for fixed-density images. Vector images can be rendered at their requested resolution (taking into account screen DPI scaling) without pixel-based rescaling. This PR does exactly that.

For Vector Image, I thought of a way to use requestWidth, requestHeight in javafx Image class constructor.

public Image(String url,
  double requestedWidth,
  double requestedHeight,
  boolean preserveRatio,
  boolean smooth,
  boolean backgroundLoading)

That's exactly what this PR gives you. If you load an SVG image using this constructor, you'll get an image that is rendered at exactly your requested resolution (again, taking into account DPI scaling).

Monitoring the size of the ImageView and calling setImage would probably work with the least amount of memory required.

That's out of scope for this feature proposal, and not something I would be in favor of.

[bridgekeeper]

@mstr2 This pull request has been inactive for more than 8 weeks and will be automatically closed if another 8 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

[bridgekeeper]

@mstr2 This pull request has been inactive for more than 16 weeks and will now be automatically closed. If you would like to continue working on this pull request in the future, feel free to reopen it! This can be done using the /open pull request command.

[mstr2]

/open

[openjdk]

@mstr2 This pull request is now open

[bridgekeeper]

@mstr2 This pull request has been inactive for more than 8 weeks and will be automatically closed if another 8 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

[bridgekeeper]

@mstr2 This pull request has been inactive for more than 16 weeks and will now be automatically closed. If you would like to continue working on this pull request in the future, feel free to reopen it! This can be done using the /open pull request command.

[mstr2]

I've changed the implementation such that this feature does not lock JavaFX into depending on java.desktop. For applications that require java.desktop (which as of now, are all JavaFX applications), the feature will be available. If we remove the java.desktop dependency in the future (by making it a static dependency), the feature will simply not be available.

I have prepared a small sample application that showcases how the feature can be used to load SVG images in a JavaFX application: https://github.com/mstr2/jfx-imageio-sample

I think this is the only way that we're ever going to get SVG support in JavaFX. Let's face it: the OpenJFX project will never be able to create its own SVG rasterizer, and it is very unlikely to take on such a huge dependency on a third-party implementation. Supporting the ImageIO API (which has an established ecosystem of image loader plugins) is the only way this highly-requested feature can realistically make its way into JavaFX.

Creating a new, bespoke JavaFX API for image loader plugins is a non-starter, as it is very unlikely that there is enough community interest in actually creating the plugins in the first place.

/open

[mstr2]

/open

[mstr2]

It seems like the bot doesn't want to reopen this PR...

[kevinrushforth]

Very odd. I'll alert the Skara folks, and reopen it manually.

[kevinrushforth]

Reopen.

[kevinrushforth]

Ah, this isn't a Skara problem. GitHub won't reopen it, when I hover over the "Reopen and comment" button, I see a tootip with "The feature/ximageloader branch was force-pushed or recreated."

I recommend creating a new Draft PR from your branch.

[kevinrushforth]

Add a comment in the new PR pointing to this PR for background information.