Skip to content

Conversation

@zsfelfoldi
Copy link
Contributor

@zsfelfoldi zsfelfoldi commented Apr 28, 2025

This PR fixes the out-of-range block number logic of getBlockLvPointer which sometimes caused searches to fail if the head was updated in the wrong moment. This logic ensures that querying the pointer of a future block returns the pointer after the last fully indexed block (instead of failing) and therefore an async range update will not cause the search to fail. Earier this behaviour only worked when headIndexed was true and headDelimiter pointed to the end of the indexed range. Now it also works for an unfinished index.

This logic is also moved from FilterMaps.getBlockLvPointer to FilterMapsMatcherBackend.GetBlockLvPointer because it is only required by the search anyways. FilterMaps.getBlockLvPointer now only returns a pointer for existing blocks, consistently with how it is used in the indexer/renderer.

Note that this unhandled case has been present in the code for a long time but went unnoticed because either one of two previously fixed bugs did prevent it from being triggered; the incorrectly positive tempRange.headIndexed (fixed in #31680), though caused other problems, prevented this one from being triggered as with a positive headIndexed no database read was triggered in getBlockLvPointer. Also, the unnecessary indexLock in synced() (fixed in #31708) usually did prevent the search seeing the temp range and therefore avoided noticeable issues.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we return the starting position of log value of the next map, if the current chain is not fully indexed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed it makes more sense to just return the start pointer of the last block.

@zsfelfoldi zsfelfoldi added this to the 1.15.11 milestone May 1, 2025
if fm.f.indexedRange.blocks.Count() > 0 {
// return index at the beginning of the last, partially indexed
// block (after the last fully indexed one)
blockNumber = fm.f.indexedRange.blocks.Last()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correct, if the requested block beyonds the last indexed block,
it's only possible that:

  • When the matcher backend is constructed, the search range is initialized with the old indexed range (e.g., 1->10)
  • Chain reorg occurs, the indexer unwinds a part of indexed range (e.g., 1->8)
  • Matcher backend is not aware of this chain reorg, and block 9, 10 is still requested for the logValue
  • The matching result is returned within the range of 1->8
  • The extra match range 9->10 will be trimmed by
// discard everything that might be invalid
trimRange := s.syncRange.ValidBlocks.Intersection(s.chainView.SharedRange(s.syncRange.IndexedView))
matchRange, matches := s.trimMatches(trimRange, r, results)
  • the extra range 9->10 will be searched again, either in indexed way or unindexed way later

defer fm.f.indexLock.RUnlock()

if blockNumber >= fm.f.indexedRange.blocks.AfterLast() {
if fm.f.indexedRange.headIndexed {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the indexed range is fully indexed, the whole range will be available for searching;

If the indexed range is partially indexed (the last one is not fully indexed), then [first, last) will be availabel for searching;

@zsfelfoldi zsfelfoldi merged commit 341929a into ethereum:master May 2, 2025
3 of 4 checks passed
@abo-L
Copy link

abo-L commented May 8, 2025

This PR fixes the out-of-range block number logic of getBlockLvPointer which sometimes caused searches to fail if the head was updated in the wrong moment. This logic ensures that querying the pointer of a future block returns the pointer after the last fully indexed block (instead of failing) and therefore an async range update will not cause the search to fail. Earier this behaviour only worked when headIndexed was true and headDelimiter pointed to the end of the indexed range. Now it also works for an unfinished index.

This logic is also moved from FilterMaps.getBlockLvPointer to FilterMapsMatcherBackend.GetBlockLvPointer because it is only required by the search anyways. FilterMaps.getBlockLvPointer now only returns a pointer for existing blocks, consistently with how it is used in the indexer/renderer.

Note that this unhandled case has been present in the code for a long time but went unnoticed because either one of two previously fixed bugs did prevent it from being triggered; the incorrectly positive tempRange.headIndexed (fixed in #31680), though caused other problems, prevented this one from being triggered as with a positive headIndexed no database read was triggered in getBlockLvPointer. Also, the unnecessary indexLock in synced() (fixed in #31708) usually did prevent the search seeing the temp range and therefore avoided noticeable issues.

jakub-freebit pushed a commit to fblch/go-ethereum that referenced this pull request Jul 3, 2025
This PR fixes the out-of-range block number logic of `getBlockLvPointer`
which sometimes caused searches to fail if the head was updated in the
wrong moment. This logic ensures that querying the pointer of a future
block returns the pointer after the last fully indexed block (instead of
failing) and therefore an async range update will not cause the search
to fail. Earier this behaviour only worked when `headIndexed` was true
and `headDelimiter` pointed to the end of the indexed range. Now it also
works for an unfinished index.

This logic is also moved from `FilterMaps.getBlockLvPointer` to
`FilterMapsMatcherBackend.GetBlockLvPointer` because it is only required
by the search anyways. `FilterMaps.getBlockLvPointer` now only returns a
pointer for existing blocks, consistently with how it is used in the
indexer/renderer.

Note that this unhandled case has been present in the code for a long
time but went unnoticed because either one of two previously fixed bugs
did prevent it from being triggered; the incorrectly positive
`tempRange.headIndexed` (fixed in
ethereum#31680), though caused other
problems, prevented this one from being triggered as with a positive
`headIndexed` no database read was triggered in `getBlockLvPointer`.
Also, the unnecessary `indexLock` in `synced()` (fixed in
ethereum#31708) usually did prevent
the search seeing the temp range and therefore avoided noticeable
issues.
howjmay pushed a commit to iotaledger/go-ethereum that referenced this pull request Aug 27, 2025
This PR fixes the out-of-range block number logic of `getBlockLvPointer`
which sometimes caused searches to fail if the head was updated in the
wrong moment. This logic ensures that querying the pointer of a future
block returns the pointer after the last fully indexed block (instead of
failing) and therefore an async range update will not cause the search
to fail. Earier this behaviour only worked when `headIndexed` was true
and `headDelimiter` pointed to the end of the indexed range. Now it also
works for an unfinished index.

This logic is also moved from `FilterMaps.getBlockLvPointer` to
`FilterMapsMatcherBackend.GetBlockLvPointer` because it is only required
by the search anyways. `FilterMaps.getBlockLvPointer` now only returns a
pointer for existing blocks, consistently with how it is used in the
indexer/renderer.

Note that this unhandled case has been present in the code for a long
time but went unnoticed because either one of two previously fixed bugs
did prevent it from being triggered; the incorrectly positive
`tempRange.headIndexed` (fixed in
ethereum#31680), though caused other
problems, prevented this one from being triggered as with a positive
`headIndexed` no database read was triggered in `getBlockLvPointer`.
Also, the unnecessary `indexLock` in `synced()` (fixed in
ethereum#31708) usually did prevent
the search seeing the temp range and therefore avoided noticeable
issues.
gballet pushed a commit to gballet/go-ethereum that referenced this pull request Sep 11, 2025
This PR fixes the out-of-range block number logic of `getBlockLvPointer`
which sometimes caused searches to fail if the head was updated in the
wrong moment. This logic ensures that querying the pointer of a future
block returns the pointer after the last fully indexed block (instead of
failing) and therefore an async range update will not cause the search
to fail. Earier this behaviour only worked when `headIndexed` was true
and `headDelimiter` pointed to the end of the indexed range. Now it also
works for an unfinished index.

This logic is also moved from `FilterMaps.getBlockLvPointer` to
`FilterMapsMatcherBackend.GetBlockLvPointer` because it is only required
by the search anyways. `FilterMaps.getBlockLvPointer` now only returns a
pointer for existing blocks, consistently with how it is used in the
indexer/renderer.

Note that this unhandled case has been present in the code for a long
time but went unnoticed because either one of two previously fixed bugs
did prevent it from being triggered; the incorrectly positive
`tempRange.headIndexed` (fixed in
ethereum#31680), though caused other
problems, prevented this one from being triggered as with a positive
`headIndexed` no database read was triggered in `getBlockLvPointer`.
Also, the unnecessary `indexLock` in `synced()` (fixed in
ethereum#31708) usually did prevent
the search seeing the temp range and therefore avoided noticeable
issues.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants