Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
90f85a0
refactor: update GraalVM native-image configuration to fix native tes…
kkriske Dec 27, 2022
306433e
refactor: use disabled annotation instead of string reference in pom …
kkriske Jan 5, 2023
f06247d
refactor: use a native-image feature instead of jni-config.json
kkriske Jan 25, 2023
0c27547
fix(jni) fix makefile linker opts to include -lm and -pthreads
Jan 28, 2023
87634c8
chore: update native libraries
github-actions[bot] Jan 31, 2023
771e205
refactor: fix multirelease jar usage in native tests
kkriske Jan 31, 2023
5b622fe
refactor: remove resource-config.json
kkriske Jan 31, 2023
e549d37
refactor: change graal-sdk dependency to provided scope
kkriske Feb 5, 2023
e4b3ebc
Merge branch 'xerial:master' into use-native-image-feature
kkriske Feb 10, 2023
386d526
refactor: leverage build-time class initialization to prevent packagi…
kkriske Feb 12, 2023
dacc01f
Merge branch 'master' into use-native-image-feature
kkriske Mar 12, 2023
c1345ed
refactor: use existing logic to determine the library to include in n…
kkriske Mar 12, 2023
a9e02a1
docs: add native-image support in README.md and document the recommen…
kkriske Mar 12, 2023
1704db6
docs: fix typo and clarify some context
kkriske Mar 12, 2023
c4dddf0
Merge branch 'master' into use-native-image-feature
kkriske Mar 24, 2023
f204385
Merge branch 'master' into use-native-image-feature
kkriske Apr 19, 2023
d9c8dc8
ci: add alternative runtime option to integration tests
kkriske Apr 21, 2023
18ebf41
ci: add separate maven profile setups for different native-image test…
kkriske Apr 24, 2023
d77b610
ci: fix syntax issue
kkriske Apr 24, 2023
cd08661
fix: revert android setup
kkriske Apr 24, 2023
e9d3aad
ci: run integration-test target instead of verify to run the test-sui…
kkriske Apr 28, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,11 +65,17 @@ jobs:
run: mvn --batch-mode --no-transfer-progress test

test_graalvm:
name: test ${{ matrix.os }} jdk${{ matrix.java }} GraalVM native-image
name: test ${{ matrix.os }} jdk${{ matrix.java }} GraalVM native-image - ${{ matrix.profiles }}
strategy:
matrix:
os: [ ubuntu-latest, windows-latest, macos-latest ]
java: [ 11, 17 ]
profiles: ['native', 'native,native-exported']
exclude: # Exclusions can be removed from GraalVM 23.0: https://github.com/oracle/graal/pull/5932
- os: ubuntu-latest
profiles: 'native,native-exported'
- os: macos-latest
profiles: 'native,native-exported'
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v3
Expand All @@ -80,7 +86,7 @@ jobs:
components: 'native-image'
github-token: ${{ secrets.GITHUB_TOKEN }}
- name: Test
run: mvn --batch-mode --no-transfer-progress -P native test
run: mvn --batch-mode --no-transfer-progress -P ${{ matrix.profiles }} integration-test

test_multiarch:
name: test ${{ matrix.arch }} ${{ matrix.distro }} jdk${{ matrix.java }}
Expand Down
37 changes: 36 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ id = 2

# How does SQLiteJDBC work?
Our SQLite JDBC driver package (i.e., `sqlite-jdbc-(VERSION).jar`) contains three
types of native SQLite libraries (`sqlite-jdbc.dll`, `sqlite-jdbc.jnilib`, `sqlite-jdbc.so`),
types of native SQLite libraries (`sqlitejdbc.dll`, `sqlitejdbc.jnilib`, `sqlitejdbc.so`),
each of them is compiled for Windows, macOS and Linux. An appropriate native library
file is automatically extracted into your OS's temporary folder, when your program
loads `org.sqlite.JDBC` driver.
Expand All @@ -114,6 +114,41 @@ In the other OSs not listed above, the pure-java SQLite is used. (Applies to ver

If you want to use the native library for your OS, [build the source from scratch](./CONTRIBUTING.md).

## GraalVM native-image support

Sqlite JDBC supports [GraalVM native-image](https://www.graalvm.org/native-image/) out of the box starting from version 3.40.1.0.
There has been rudimentary support for some versions before that, but this was not actively tested by the CI.

By default, the `sqlitejdbc` library for the compilation target will be included in the native image, accompanied by the required JNI configuration.
At runtime, this library will be extracted to the temp folder and loaded from there.
For faster startup however, it is recommended to set the `org.sqlite.lib.exportPath` property at build-time.
This will export the `sqlitejdbc` library at build-time to the specified directory, and the library will not be included as a resource.
As a result, the native image itself will be slightly smaller and the overhead of exporting the library at run-time is eliminated,
but you need to make sure the library can be found at run-time.
The best way to do this is to simply place the library next to the executable.

### CLI example
```shell
native-image -Dorg.sqlite.lib.exportPath=~/outDir -H:Path=~/outDir -cp foo.jar org.example.Main
```
This will place both the `sqlitejdbc` shared library and the native-image output in the `~/outDir` folder.

### Maven example
This example uses the [native-build-tools](https://graalvm.github.io/native-build-tools/latest/index.html) maven plugin:
```xml
<plugin>
<groupId>org.graalvm.buildtools</groupId>
<artifactId>native-maven-plugin</artifactId>
<configuration>
<buildArgs>
<buildArg>-Dorg.sqlite.lib.exportPath=${project.build.directory}</buildArg>
</buildArgs>
</configuration>
</plugin>
```
This will automatically place the `sqlitejdbc` library in the `/target` folder of your project, creating a functional execution environment.
When packaging the resulting app, simply include the library in the distribution bundle.

# Download

Download from [Maven Central](https://search.maven.org/artifact/org.xerial/sqlite-jdbc) or from the [releases](https://github.com/xerial/sqlite-jdbc/releases) page.
Expand Down
34 changes: 32 additions & 2 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<junit.version>5.9.2</junit.version>
<surefire.version>3.0.0</surefire.version>
<graalvm.version>22.3.0</graalvm.version>
<java9.sourceDirectory>${project.basedir}/src/main/java9</java9.sourceDirectory>
</properties>

Expand Down Expand Up @@ -340,11 +341,11 @@
<extensions>true</extensions>
<executions>
<execution>
<id>test-native</id>
<id>test-native-default</id>
<goals>
<goal>test</goal>
</goals>
<phase>test</phase>
<phase>integration-test</phase>
</execution>
</executions>
<configuration>
Expand Down Expand Up @@ -378,9 +379,38 @@
</plugins>
</build>
</profile>
<profile>
<id>native-exported</id>
<build>
<plugins>
<plugin>
<groupId>org.graalvm.buildtools</groupId>
<artifactId>native-maven-plugin</artifactId>
<configuration>
<buildArgs combine.children="append">
<buildArg>
-Dorg.sqlite.lib.exportPath=${project.build.directory}
</buildArg>
</buildArgs>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>

<dependencies>
<!--
This dependency makes compilation on non-GraalVM versions possible.
The dependency should however never actually be required by end-users because if they require
GraalVM features, they will be provided by the JDK they are using.
-->
<dependency>
<groupId>org.graalvm.sdk</groupId>
<artifactId>graal-sdk</artifactId>
<version>${graalvm.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
Expand Down
82 changes: 39 additions & 43 deletions src/main/java/org/sqlite/SQLiteJDBCLoader.java
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,10 @@
import java.io.*;
import java.net.URL;
import java.net.URLConnection;
import java.nio.file.*;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
import java.security.DigestInputStream;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
Expand All @@ -36,6 +39,7 @@
import java.util.Properties;
import java.util.UUID;
import java.util.stream.Stream;
import org.sqlite.util.LibraryLoaderUtil;
import org.sqlite.util.OSInfo;
import org.sqlite.util.StringUtils;

Expand Down Expand Up @@ -310,10 +314,7 @@ private static void loadSQLiteNativeLibrary() throws Exception {
String sqliteNativeLibraryPath = System.getProperty("org.sqlite.lib.path");
String sqliteNativeLibraryName = System.getProperty("org.sqlite.lib.name");
if (sqliteNativeLibraryName == null) {
sqliteNativeLibraryName = System.mapLibraryName("sqlitejdbc");
if (sqliteNativeLibraryName != null && sqliteNativeLibraryName.endsWith(".dylib")) {
sqliteNativeLibraryName = sqliteNativeLibraryName.replace(".dylib", ".jnilib");
}
sqliteNativeLibraryName = LibraryLoaderUtil.getNativeLibName();
}

if (sqliteNativeLibraryPath != null) {
Expand All @@ -326,22 +327,9 @@ private static void loadSQLiteNativeLibrary() throws Exception {
}

// Load the os-dependent library from the jar file
String packagePath = SQLiteJDBCLoader.class.getPackage().getName().replaceAll("\\.", "/");
sqliteNativeLibraryPath =
String.format(
"/%s/native/%s", packagePath, OSInfo.getNativeLibFolderPathForCurrentOS());
boolean hasNativeLib = hasResource(sqliteNativeLibraryPath + "/" + sqliteNativeLibraryName);

if (!hasNativeLib) {
if (OSInfo.getOSName().equals("Mac")) {
// Fix for openjdk7 for Mac
String altName = "libsqlitejdbc.jnilib";
if (hasResource(sqliteNativeLibraryPath + "/" + altName)) {
sqliteNativeLibraryName = altName;
hasNativeLib = true;
}
}
}
sqliteNativeLibraryPath = LibraryLoaderUtil.getNativeLibResourcePath();
boolean hasNativeLib =
LibraryLoaderUtil.hasNativeLib(sqliteNativeLibraryPath, sqliteNativeLibraryName);

if (hasNativeLib) {
// temporary library folder
Expand Down Expand Up @@ -379,10 +367,6 @@ private static void loadSQLiteNativeLibrary() throws Exception {
StringUtils.join(triedPaths, File.pathSeparator)));
}

private static boolean hasResource(String path) {
return SQLiteJDBCLoader.class.getResource(path) != null;
}

@SuppressWarnings("unused")
private static void getNativeLibraryFolderForTheCurrentOS() {
String osName = OSInfo.getOSName();
Expand All @@ -403,27 +387,39 @@ public static int getMinorVersion() {

/** @return The version of the SQLite JDBC driver. */
public static String getVersion() {
return VersionHolder.VERSION;
}

URL versionFile =
SQLiteJDBCLoader.class.getResource(
"/META-INF/maven/org.xerial/sqlite-jdbc/pom.properties");
if (versionFile == null) {
versionFile =
SQLiteJDBCLoader.class.getResource(
"/META-INF/maven/org.xerial/sqlite-jdbc/VERSION");
}
/**
* This class will load the version from resources during <clinit>. By initializing this at
* build-time in native-image, the resources do not need to be included in the native
* executable, and we're eliminating the IO operations as well.
*/
public static final class VersionHolder {
private static final String VERSION;

static {
URL versionFile =
VersionHolder.class.getResource(
"/META-INF/maven/org.xerial/sqlite-jdbc/pom.properties");
if (versionFile == null) {
versionFile =
VersionHolder.class.getResource(
"/META-INF/maven/org.xerial/sqlite-jdbc/VERSION");
}

String version = "unknown";
try {
if (versionFile != null) {
Properties versionData = new Properties();
versionData.load(versionFile.openStream());
version = versionData.getProperty("version", version);
version = version.trim().replaceAll("[^0-9\\.]", "");
String version = "unknown";
try {
if (versionFile != null) {
Properties versionData = new Properties();
versionData.load(versionFile.openStream());
version = versionData.getProperty("version", version);
version = version.trim().replaceAll("[^0-9\\.]", "");
}
} catch (IOException e) {
e.printStackTrace();
}
} catch (IOException e) {
e.printStackTrace();
VERSION = version;
}
return version;
}
}
28 changes: 28 additions & 0 deletions src/main/java/org/sqlite/util/LibraryLoaderUtil.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
package org.sqlite.util;

import org.sqlite.SQLiteJDBCLoader;

public class LibraryLoaderUtil {
/**
* Get the OS-specific resource directory within the jar, where the relevant sqlitejdbc native
* library is located.
*/
public static String getNativeLibResourcePath() {
String packagePath = SQLiteJDBCLoader.class.getPackage().getName().replace(".", "/");
return String.format(
"/%s/native/%s", packagePath, OSInfo.getNativeLibFolderPathForCurrentOS());
}

/** Get the OS-specific name of the sqlitejdbc native library. */
public static String getNativeLibName() {
String nativeLibName = System.mapLibraryName("sqlitejdbc");
if (nativeLibName != null && nativeLibName.endsWith(".dylib")) {
nativeLibName = nativeLibName.replace(".dylib", ".jnilib");
}
return nativeLibName;
}

public static boolean hasNativeLib(String path, String libraryName) {
return SQLiteJDBCLoader.class.getResource(path + "/" + libraryName) != null;
}
}
1 change: 1 addition & 0 deletions src/main/java9/module-info.java
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

requires transitive java.sql;
requires transitive java.sql.rowset;
requires static org.graalvm.sdk;

exports org.sqlite;
exports org.sqlite.core;
Expand Down
Loading