progcache: support concurrent usage #6653

ripatel-fd · 2025-10-07T22:46:56Z

No description provided.

github-actions · 2025-10-13T02:10:28Z

Performance Measurements ⏳

Suite	Baseline	New	Change
backtest `mainnet-368528500-perf` per slot	`0.083989 s`	`0.082585 s`	`-1.672%` ✅
backtest `mainnet-368528500-perf` snapshot load	`2.927 s`	`2.901 s`	`-0.888%` ✅
backtest `mainnet-368528500-perf` total elapsed	`83.98879 s`	`82.584563 s`	`-1.672%` ✅
`firedancer mem` usage with `mainnet.toml`	`1014.12 GiB`	`1031.12 GiB`	`1.676%` ✅

github-actions · 2025-10-13T15:53:02Z

Performance Measurements ⏳

Suite	Baseline	New	Change
backtest `mainnet-368528500-perf` per slot	`0.081677 s`	`0.080741 s`	`-1.146%` ✅
backtest `mainnet-368528500-perf` snapshot load	`2.889 s`	`2.882 s`	`-0.242%` ✅
backtest `mainnet-368528500-perf` total elapsed	`81.677232 s`	`80.74117 s`	`-1.146%` ✅
`firedancer mem` usage with `mainnet.toml`	`1014.12 GiB`	`1031.12 GiB`	`1.676%` ✅

github-actions · 2025-10-13T16:34:37Z

Performance Measurements ⏳

Suite	Baseline	New	Change
backtest `mainnet-368528500-perf` per slot	`0.082107 s`	`0.081033 s`	`-1.308%` ✅
backtest `mainnet-368528500-perf` snapshot load	`2.902 s`	`2.912 s`	`0.345%` ✅
backtest `mainnet-368528500-perf` total elapsed	`82.106703 s`	`81.032858 s`	`-1.308%` ✅
`firedancer mem` usage with `mainnet.toml`	`1014.12 GiB`	`1017.12 GiB`	`0.296%` ✅

github-actions · 2025-10-13T18:12:56Z

Performance Measurements ⏳

Suite	Baseline	New	Change
backtest `mainnet-368528500-perf` per slot	`0.083679 s`	`0.082533 s`	`-1.370%` ✅
backtest `mainnet-368528500-perf` snapshot load	`2.94 s`	`2.936 s`	`-0.136%` ✅
backtest `mainnet-368528500-perf` total elapsed	`83.679333 s`	`82.533329 s`	`-1.370%` ✅
`firedancer mem` usage with `mainnet.toml`	`1014.12 GiB`	`1017.12 GiB`	`0.296%` ✅

github-actions · 2025-10-13T18:47:44Z

Performance Measurements ⏳

Suite	Baseline	New	Change
backtest `mainnet-368528500-perf` per slot	`0.083729 s`	`0.082377 s`	`-1.615%` ✅
backtest `mainnet-368528500-perf` snapshot load	`2.925 s`	`2.904 s`	`-0.718%` ✅
backtest `mainnet-368528500-perf` total elapsed	`83.728786 s`	`82.377332 s`	`-1.614%` ✅
`firedancer mem` usage with `mainnet.toml`	`1014.12 GiB`	`1017.12 GiB`	`0.296%` ✅

github-actions · 2025-10-13T18:58:12Z

Performance Measurements ⏳

Suite	Baseline	New	Change
backtest `mainnet-368528500-perf` per slot	`0.08249 s`	`0.081032 s`	`-1.767%` ✅
backtest `mainnet-368528500-perf` snapshot load	`2.895 s`	`2.901 s`	`0.207%` ✅
backtest `mainnet-368528500-perf` total elapsed	`82.490041 s`	`81.032281 s`	`-1.767%` ✅
`firedancer mem` usage with `mainnet.toml`	`1014.12 GiB`	`1017.12 GiB`	`0.296%` ✅

mjain-jump · 2025-10-13T20:47:15Z

src/flamenco/progcache/fd_progcache_user.c

+  fd_progcache_load_fork_slow( cache, xid, epoch_slot0 ); /* switch fork */
+}
+
+/* fd_progcache_query searches for a program cache entry on the current


stray comment

no it's supposed to be there

i see, but it looks like you used the wrong function name in your documentation (should be fd_progcache_fork_has_xid?)

This comment refers to the query routine overall, but some parts of query (like fork_has_xid) is split out into separate functions.

src/flamenco/progcache/fd_progcache_user.h

src/flamenco/progcache/fd_progcache_user.c

mjain-jump · 2025-10-13T20:58:50Z

src/flamenco/progcache/fd_progcache_user.c

+  fd_progcache_rec_t * rec = fd_progcache_rec_new_nx( rec_mem, slot );
+  rec->invalidate = 1;


whats the purpose of having both invalidate and executable? can we use one or the other?

if i understand correctly, its pretty much the equivalent of the last_modified_slot and failed_verification flags from the previous implementation?

@mjain-jump To be honest, I didn't really understand what "last_modified_slot" does, since it was zero most of the time.

invalidate means: "This program was modified at the current fork graph node, you need to reload it in a future slot"

there's a list called programs_to_reverify inside the txn ctx that gets amended every time fd_deploy_program is called (for upgrades / deployments). in txn finalize we iterate over this list and set the last_slot_modified field to the current slot for each of the cache entries. so i think yes it's pretty much equivalent to your invalidate flag. what's the purpose of executable in your PR?

what's the purpose of executable in your PR?

This is documented

Entries are either non-executable (e.g. programs that failed
verification) or executable. Non-executable entry objects consist
only of this header struct. Executable entry objects are variable-
sized and contain additional structures past this header (rodata/ROM
segment, control flow metadata, ...).
[...]
uint executable : 1; /* is this an executable entry? */

src/flamenco/runtime/program/fd_bpf_loader_program.c

github-actions · 2025-10-13T22:13:50Z

Performance Measurements ⏳

Suite	Baseline	New	Change
backtest `mainnet-368528500-perf` per slot	`0.082259 s`	`0.080927 s`	`-1.619%` ✅
backtest `mainnet-368528500-perf` snapshot load	`2.918 s`	`2.903 s`	`-0.514%` ✅
backtest `mainnet-368528500-perf` total elapsed	`82.258801 s`	`80.927384 s`	`-1.619%` ✅
`firedancer mem` usage with `mainnet.toml`	`1014.12 GiB`	`1017.12 GiB`	`0.296%` ✅

github-actions · 2025-10-13T22:26:13Z

Performance Measurements ⏳

Suite	Baseline	New	Change
backtest `mainnet-368528500-perf` per slot	`0.082222 s`	`0.081026 s`	`-1.455%` ✅
backtest `mainnet-368528500-perf` snapshot load	`2.925 s`	`2.892 s`	`-1.128%` ✅
backtest `mainnet-368528500-perf` total elapsed	`82.221747 s`	`81.025683 s`	`-1.455%` ✅
`firedancer mem` usage with `mainnet.toml`	`1014.12 GiB`	`1017.12 GiB`	`0.296%` ✅

github-actions · 2025-10-14T06:23:18Z

Performance Measurements ⏳

Suite	Baseline	New	Change
backtest `mainnet-368528500-perf` per slot	`0.08216 s`	`0.081254 s`	`-1.103%` ✅
backtest `mainnet-368528500-perf` snapshot load	`2.931 s`	`2.904 s`	`-0.921%` ✅
backtest `mainnet-368528500-perf` total elapsed	`82.160301 s`	`81.253941 s`	`-1.103%` ✅
`firedancer mem` usage with `mainnet.toml`	`1014.12 GiB`	`1017.12 GiB`	`0.296%` ✅

github-actions · 2025-10-14T16:14:26Z

Performance Measurements ⏳

Suite	Baseline	New	Change
backtest `mainnet-368528500-perf` per slot	`0.082092 s`	`0.080903 s`	`-1.448%` ✅
backtest `mainnet-368528500-perf` snapshot load	`2.964 s`	`2.896 s`	`-2.294%` ✅
backtest `mainnet-368528500-perf` total elapsed	`82.091857 s`	`80.902868 s`	`-1.448%` ✅
`firedancer mem` usage with `mainnet.toml`	`1014.12 GiB`	`1017.12 GiB`	`0.296%` ✅

github-actions · 2025-10-14T17:17:04Z

Performance Measurements ⏳

Suite	Baseline	New	Change
backtest `mainnet-368528500-perf` per slot	`0.081915 s`	`0.080576 s`	`-1.635%` ✅
backtest `mainnet-368528500-perf` snapshot load	`2.914 s`	`2.893 s`	`-0.721%` ✅
backtest `mainnet-368528500-perf` total elapsed	`81.914563 s`	`80.576314 s`	`-1.634%` ✅
`firedancer mem` usage with `mainnet.toml`	`1014.12 GiB`	`1017.12 GiB`	`0.296%` ✅

- Reduce funk key size to 32 bytes (from 40 bytes) - Move program cache to separate funk instance and wksp - Restructure program cache logic / general cleanup - Reduce verbosity in API documentation - Clean up API naming - Split up API into a few smaller headers - Reduce program cache fill allocations and memory copies (directly populate calldests map, skip calldests map alloc for newer sBPF versions) - Remove broken/unused concurrency model in previous program cache implementation - Simplify cache invalidation logic ("queue program for re- verification"): now just inserts a tombstone - Reuse cache entries across forks (cache entries tracked at fork graph nodes determined by program modify slot, instead of program load slot) - Remove program pre-loading - Make program cache fills shared memory concurrent (in exec tiles)

github-actions · 2025-10-14T18:20:32Z

Performance Measurements ⏳

Suite	Baseline	New	Change
backtest `mainnet-368528500-perf` per slot	`0.081999 s`	`0.081197 s`	`-0.978%` ✅
backtest `mainnet-368528500-perf` snapshot load	`2.923 s`	`2.9 s`	`-0.787%` ✅
backtest `mainnet-368528500-perf` total elapsed	`81.99911 s`	`81.197497 s`	`-0.978%` ✅
`firedancer mem` usage with `mainnet.toml`	`1014.12 GiB`	`1017.12 GiB`	`0.296%` ✅

ripatel-fd · 2025-10-14T20:35:12Z

src/flamenco/progcache/fd_progcache_user.c

+    if( FD_UNLIKELY( (long)found_slot<best_slot ) ) continue;
+
+    /* Confirm that record is part of the current fork
+       FIXME this has bad performance / pointer-chasing */


can remove this FIXME, i actually addressed this problem

mmcgee-jump · 2025-10-14T20:37:23Z

src/disco/metrics/metrics.xml

+<tile name="exec">
+    <counter name="ProgcacheMisses" summary="Number of program cache misses" />
+    <counter name="ProgcacheHits" summary="Number of program cache hits" />
+    <counter name="ProgcacheFills" summary="Number of program cache insertions" />


split to enum?

ProgCacheInsertResult

Success

SuccessTombstone

Duplicate

Will do this and other suggestions discussed in chat in the next PR

ripatel-fd force-pushed the ripatel/progcache-concur branch 7 times, most recently from b353004 to 0fd9786 Compare October 8, 2025 06:39

ripatel-fd mentioned this pull request Oct 8, 2025

Rewrite program cache #6608

Closed

ripatel-fd force-pushed the ripatel/progcache-concur branch 8 times, most recently from d6f483d to 88dbee9 Compare October 13, 2025 02:04

ripatel-fd marked this pull request as ready for review October 13, 2025 02:04

ripatel-fd force-pushed the ripatel/progcache-concur branch 2 times, most recently from 185c29a to 11ac416 Compare October 13, 2025 15:26

ripatel-fd force-pushed the ripatel/progcache-concur branch from 11ac416 to 4ee9125 Compare October 13, 2025 16:24

ripatel-fd force-pushed the ripatel/progcache-concur branch from 4ee9125 to f43e815 Compare October 13, 2025 18:03

ripatel-fd force-pushed the ripatel/progcache-concur branch from f43e815 to d14cb67 Compare October 13, 2025 18:38

ripatel-fd force-pushed the ripatel/progcache-concur branch from d14cb67 to f7f0ce2 Compare October 13, 2025 18:49

ripatel-fd force-pushed the ripatel/progcache-concur branch from f7f0ce2 to 4ea6c22 Compare October 13, 2025 19:39

ripatel-fd force-pushed the ripatel/progcache-concur branch from 4ea6c22 to a93634e Compare October 13, 2025 20:40

mjain-jump reviewed Oct 13, 2025

View reviewed changes

src/flamenco/progcache/fd_progcache_user.h Show resolved Hide resolved

mjain-jump reviewed Oct 13, 2025

View reviewed changes

src/flamenco/progcache/fd_progcache_user.c Outdated Show resolved Hide resolved

mjain-jump reviewed Oct 13, 2025

View reviewed changes

src/flamenco/runtime/program/fd_bpf_loader_program.c Show resolved Hide resolved

ripatel-fd force-pushed the ripatel/progcache-concur branch 3 times, most recently from bbbcd92 to a1158ab Compare October 13, 2025 22:07

ripatel-fd force-pushed the ripatel/progcache-concur branch from a1158ab to e1a6a4f Compare October 13, 2025 22:17

ripatel-fd force-pushed the ripatel/progcache-concur branch 2 times, most recently from 95a4fd9 to c116f73 Compare October 14, 2025 05:24

ripatel-fd force-pushed the ripatel/progcache-concur branch 2 times, most recently from d8817a1 to a71b8f8 Compare October 14, 2025 16:05

ripatel-fd force-pushed the ripatel/progcache-concur branch from a71b8f8 to de1ca6c Compare October 14, 2025 17:12

ripatel-fd force-pushed the ripatel/progcache-concur branch from de1ca6c to 2a40eec Compare October 14, 2025 18:14

ripatel-fd commented Oct 14, 2025

View reviewed changes

mmcgee-jump reviewed Oct 14, 2025

View reviewed changes

mmcgee-jump approved these changes Oct 14, 2025

View reviewed changes

ripatel-fd added this pull request to the merge queue Oct 14, 2025

Merged via the queue into main with commit 2dca3a2 Oct 14, 2025
12 checks passed

ripatel-fd deleted the ripatel/progcache-concur branch October 14, 2025 21:04

		fd_progcache_rec_t * rec = fd_progcache_rec_new_nx( rec_mem, slot );
		rec->invalidate = 1;

progcache: support concurrent usage #6653

progcache: support concurrent usage #6653

Uh oh!

Conversation

ripatel-fd commented Oct 7, 2025

Uh oh!

github-actions bot commented Oct 13, 2025

Performance Measurements ⏳

Uh oh!

github-actions bot commented Oct 13, 2025

Performance Measurements ⏳

Uh oh!

github-actions bot commented Oct 13, 2025

Performance Measurements ⏳

Uh oh!

github-actions bot commented Oct 13, 2025

Performance Measurements ⏳

Uh oh!

github-actions bot commented Oct 13, 2025

Performance Measurements ⏳

Uh oh!

github-actions bot commented Oct 13, 2025

Performance Measurements ⏳

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Oct 13, 2025

Performance Measurements ⏳

Uh oh!

github-actions bot commented Oct 13, 2025

Performance Measurements ⏳

Uh oh!

github-actions bot commented Oct 14, 2025

Performance Measurements ⏳

Uh oh!

github-actions bot commented Oct 14, 2025

Performance Measurements ⏳

Uh oh!

github-actions bot commented Oct 14, 2025

Performance Measurements ⏳

Uh oh!

github-actions bot commented Oct 14, 2025

Performance Measurements ⏳

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants