Skip to content

Conversation

@holiman
Copy link
Collaborator

@holiman holiman commented Sep 2, 2020

These changes reduces the allocs and allocated bytes (heap allocs) quite substantially.

name                       old time/op    new time/op    delta
DeriveSha200/std_trie-6       565µs ± 8%     590µs ±16%     ~     (p=0.310 n=5+5)
DeriveSha200/stack_trie-6     534µs ±11%     452µs ± 4%  -15.42%  (p=0.008 n=5+5)

name                       old alloc/op   new alloc/op   delta
DeriveSha200/std_trie-6       274kB ± 0%     274kB ± 0%   +0.02%  (p=0.008 n=5+5)
DeriveSha200/stack_trie-6     111kB ± 0%      56kB ± 0%  -49.73%  (p=0.008 n=5+5)

name                       old allocs/op  new allocs/op  delta
DeriveSha200/std_trie-6       2.86k ± 0%     2.86k ± 0%     ~     (all equal)
DeriveSha200/stack_trie-6     1.86k ± 0%     1.43k ± 0%  -22.95%  (p=0.008 n=5+5)

But I figured I'd rather make it as a PR than push to your branch, because the changes might be 'ugly' ...

// Add new child
if st.children[idx] == nil {
st.children[idx] = stackTrieFromPool(st.db)
st.children[idx].keyOffset = st.keyOffset + 1
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any reason this got moved here?

st.children[idx] = NewStackTrie(st.db)
st.children[idx].keyOffset = st.keyOffset + 1
}
// Unresolve elder siblings
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer "hash" to "unresolve"

@holiman
Copy link
Collaborator Author

holiman commented Sep 3, 2020 via email

@gballet gballet merged commit a1300e6 into gballet:derive-sha-with-stacktrie-rewrite Sep 3, 2020
gballet added a commit that referenced this pull request Sep 24, 2020
core/types: use stacktrie for derivesha

trie: add stacktrie file

trie: fix linter

core/types: use stacktrie for derivesha

rebased: adapt stacktrie to the newer version of DeriveSha

Co-authored-by: Martin Holst Swende <[email protected]>

More linter fixes

review feedback: no key offset for nodes converted to hashes

trie: use EncodeRLP for full nodes

core/types: insert txs in order in derivesha

trie: tests for derivesha with stacktrie

trie: make stacktrie use pooled hashers

trie: make stacktrie reuse tmp slice space

trie: minor polishes on stacktrie

trie/stacktrie: less rlp dancing

core/types: explain the contorsions in DeriveSha

ci: fix goimport errors

trie: clear mem on subtrie hashing

squashme: linter fix

stracktrie: use pooling, less allocs (#3)

trie: in-place hex prefix, reduce allocs and add rawNode.EncodeRLP

Reintroduce the `[]node` method, add the missing `EncodeRLP` implementation for `rawNode` and calculate the hex prefix in place.

Co-authored-by: Martin Holst Swende <[email protected]>
gballet added a commit that referenced this pull request Oct 3, 2020
core/types: use stacktrie for derivesha

trie: add stacktrie file

trie: fix linter

core/types: use stacktrie for derivesha

rebased: adapt stacktrie to the newer version of DeriveSha

Co-authored-by: Martin Holst Swende <[email protected]>

More linter fixes

review feedback: no key offset for nodes converted to hashes

trie: use EncodeRLP for full nodes

core/types: insert txs in order in derivesha

trie: tests for derivesha with stacktrie

trie: make stacktrie use pooled hashers

trie: make stacktrie reuse tmp slice space

trie: minor polishes on stacktrie

trie/stacktrie: less rlp dancing

core/types: explain the contorsions in DeriveSha

ci: fix goimport errors

trie: clear mem on subtrie hashing

squashme: linter fix

stracktrie: use pooling, less allocs (#3)

trie: in-place hex prefix, reduce allocs and add rawNode.EncodeRLP

Reintroduce the `[]node` method, add the missing `EncodeRLP` implementation for `rawNode` and calculate the hex prefix in place.

Co-authored-by: Martin Holst Swende <[email protected]>

Co-authored-by: Martin Holst Swende <[email protected]>
gballet added a commit that referenced this pull request Dec 24, 2023
gballet added a commit that referenced this pull request Mar 15, 2024
regenerate posdao contract bindings

more build fixes

more build fixes

more build fixes

fix simple name

fix more build errors

39 build errors to go!

fix build

add chiado chainspec

start import from #3

forkid hack

genesis init

base fee collector

only apply max code size check after shanghai

handshake hack

add bootnodes to list

add TTD + banner info

fix: create the correct genesis

fix: duplicate field

a few bug fixes found during my tests

more error fixes

fixes to run through up to 6M blocks and counting

fix eip1559FeeCollector location in config

config fix + support post london free txs

fix contract rewrite decoding

fix resume-time panic when still in aura mode

speedup: hardcode forkblock to determine if pos is active

add withdrawals contract to config

add cancun time
gballet added a commit that referenced this pull request Apr 7, 2024
gballet added a commit that referenced this pull request Apr 24, 2024
gballet pushed a commit that referenced this pull request Mar 28, 2025
…m#31081)

This PR is #3 of a 3-part series that implements the new log index
intended to replace core/bloombits.
Based on ethereum#31079 and
ethereum#31080
Replaces ethereum#30370

This part removes the old bloombits package and the chain indexer that
was only used by bloombits. Deletes the old bloombits database.

FilterMaps data structure explanation:
https://gist.github.com/zsfelfoldi/a60795f9da7ae6422f28c7a34e02a07e

Log index generator code overview:
https://gist.github.com/zsfelfoldi/97105dff0b1a4f5ed557924a24b9b9e7

Search pattern matcher code overview:
https://gist.github.com/zsfelfoldi/5981735641c956afb18065e84f8aff34

Note that the possibility of a tree hashing scheme and remote proof
protocol are mentioned in the documents above but they are not exactly
specified yet. These specs are WIP and will be finalized after the local
log indexer/filter code is finalized and merged.

---------

Co-authored-by: Felix Lange <[email protected]>
gballet pushed a commit that referenced this pull request Sep 11, 2025
…m#31081)

This PR is #3 of a 3-part series that implements the new log index
intended to replace core/bloombits.
Based on ethereum#31079 and
ethereum#31080
Replaces ethereum#30370

This part removes the old bloombits package and the chain indexer that
was only used by bloombits. Deletes the old bloombits database.

FilterMaps data structure explanation:
https://gist.github.com/zsfelfoldi/a60795f9da7ae6422f28c7a34e02a07e

Log index generator code overview:
https://gist.github.com/zsfelfoldi/97105dff0b1a4f5ed557924a24b9b9e7

Search pattern matcher code overview:
https://gist.github.com/zsfelfoldi/5981735641c956afb18065e84f8aff34

Note that the possibility of a tree hashing scheme and remote proof
protocol are mentioned in the documents above but they are not exactly
specified yet. These specs are WIP and will be finalized after the local
log indexer/filter code is finalized and merged.

---------

Co-authored-by: Felix Lange <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants