Skip to content

Conversation

@weary-adventurer
Copy link
Contributor

@weary-adventurer weary-adventurer commented Feb 28, 2025

This adds a benchmark for the built-in node:sqlite module available with --experimental-sqlite since Node v22.5.0.
This benchmark will only run and be included in the results if the node:sqlite module exists and is available.

Known issues:

  1. This creates prepared statements for every query instead of reusing persistent statements because DatabaseSync.prepare is a wrapper for sqlite3_prepare_v2 that doesn't accept SQLITE_PREPARE_PERSISTENT flag.
  2. Iterate benchmark does nothing for now as StatementSync.iterate was added in Node v23.4.0 but even after updating it fails with "statement has been finalized".

This is probably OK as a first pass and will be more useful later as the built-in module gets more stable.

Fixes #1266.

$ node --experimental-sqlite benchmark
--- reading rows individually ---
better-sqlite3 x 121,663 ops/sec ±0.29%
node-sqlite3   x 12,805 ops/sec ±0.47%
node:sqlite    x 52,115 ops/sec ±1.66%

--- reading 100 rows into an array ---
better-sqlite3 x 6,750 ops/sec ±0.41%
node-sqlite3   x 1,641 ops/sec ±0.8%
node:sqlite    x 10,170 ops/sec ±0.42%

--- iterating over 100 rows ---
better-sqlite3 x 5,732 ops/sec ±0.44%
node-sqlite3   x 132 ops/sec ±1.78%
node:sqlite    x NaN ops/sec ±NaN% // See [2]

--- inserting rows individually ---
better-sqlite3 x 39,821 ops/sec ±4.34%
node-sqlite3   x 11,392 ops/sec ±1.21%
node:sqlite    x 26,655 ops/sec ±2.28%

--- inserting 100 rows in a single transaction ---
better-sqlite3 x 3,996 ops/sec ±3.2%
node-sqlite3   x 105 ops/sec ±1.38%
node:sqlite    x 2,906 ops/sec ±2.71%

All benchmarks complete!

process.stdout.write(`${driver} (running...)\n`);
try {
const result = execFileSync('node', ['./benchmark.js', ctx], { stdio: 'pipe', encoding: 'utf8' });
const result = execFileSync('node', [...process.execArgv, './benchmark.js', ctx], { stdio: 'pipe', encoding: 'utf8' });
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the purpose of ...process.execArgv?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you don't pass --experimental-sqlite to node, the node:sqlite module will not be available. This makes it so node child processes get the same node-specific arguments as the parent.

const sql = `SELECT ${columns.join(', ')} FROM ${table} WHERE rowid >= ? LIMIT 100`;
let rowid = -100;
return () => {
const stmt = db.prepare(sql);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The SQLITE_PREPARE_PERSISTENT flag is just an optimization hint. You should be able to reuse prepared statements without it.


return () => {
// Error: statement has been finalized
// for (const row of db.prepare(sql).iterate((rowid += 100) % count + 1)) {}
Copy link
Member

@JoshuaWise JoshuaWise Apr 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll probably wait to merge this until node:sqlite is a little more stable. But thanks for putting this together!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I made this there were a few issues with node:sqlite. You can try to rerun it today against latest node and see if they are fixed or adjust the code if the functions are different. I think it is still better than nothing so you can at least get an idea of how the two implementations perform, even if this test does not work.

@JoshuaWise JoshuaWise merged commit b35748f into WiseLibs:master Jul 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

node:sqlite and benchmarking

2 participants