Skip to content

fix(store): harden MultiListIterator close and next - #3445

Open
vaidikgarg-bit wants to merge 1 commit into
apache:masterfrom
vaidikgarg-bit:fix/multilist-iterator-close-contract
Open

vaidikgarg-bit wants to merge 1 commit into
apache:masterfrom
vaidikgarg-bit:fix/multilist-iterator-close-contract

Conversation

@vaidikgarg-bit

@vaidikgarg-bit vaidikgarg-bit commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Purpose of the PR

#3437 made MultiListIterator.close() close every sub-iterator when it runs before the first hasNext(). Three problems remained in the same class:

  1. close() closes the sub-iterators with no try/finally. When one close() throws, the remaining sub-iterators stay open and the list is never cleared.
  2. next() dereferences innerIterator, which is only set by hasNext()/isValid(). Calling next() first, or after hasNext() has returned false, throws NullPointerException. Repeated next() calls without hasNext() also never move past the current sub-iterator.
  3. close() leaves innerIterator pointing at the source it just closed. The next hasNext() or isValid() calls into that closed source and can close it again.

Main Changes

  • Add a closed flag, as UnionFilterIterator and IntersectionFilterIterator have. Once it is set, hasNext() and isValid() return false without touching any source, and a second close() does nothing.
  • close() drops its reference to the current sub-iterator and keeps going when a sub-iterator's close() throws a RuntimeException or an Error (RocksDBScanIterator.close() rethrows both). It clears the list and then rethrows the first failure, with any later failures added as suppressed exceptions.
  • next() calls hasNext() first, so it moves on to the next sub-iterator by itself and throws NoSuchElementException when nothing is left, including after close(). With the usual hasNext()/next() loop this costs one extra hasNext() on the current sub-iterator per row (for RocksDBScanIterator, one more isOwningHandle()/isValid()/key() round trip and key filter).

Not changed here: position() has no closed check, so after closing mid-iteration it now throws NullPointerException instead of reading the closed source (before iteration it already threw on master). A close failure raised while hasNext() moves past an exhausted sub-iterator is also left as is.

Verifying these changes

  • Trivial rework / code cleanup without any test coverage. (No Need)
  • Already covered by existing tests, such as (please modify tests here).
  • Need tests and can be verified as follows:
    • five new tests in FilterIteratorContractTest: a RuntimeException close failure before iteration, an Error close failure before iteration, a close failure during iteration, next() without hasNext(), and hasNext()/isValid()/next()/close() after closing mid-iteration. All five fail on master.
    • store-server-test (202), store-core-test (113) and store-rocksdb-test (19) pass locally on 8e327b8f3, and mvn editorconfig:check is clean for the changed modules.

Does this PR potentially affect the following parts?

Documentation Status

Select one option and provide the documentation location when applicable.

  • Doc - TODO: required documentation is pending; complete it before merging.
  • Doc - Done: documentation is included here or linked below.
  • Doc - No Need: no user-visible documentation is affected.

Documentation files in this PR or paired hugegraph-doc PR:

Visual overview

MultiListIterator close and next contract

Explicit close() marks the iterator closed, detaches the current source, attempts the current and remaining source closes despite RuntimeException/Error, clears the list, and rethrows the first failure with later failures suppressed when a failure occurred. Subsequent probes stop and repeated close() is a no-op. next() checks availability itself and throws NoSuchElementException when exhausted or closed. position() and advance-time close failures remain outside this fix; the additional hasNext() call per row has no bounded performance measurement here. This illustrates the current diff, not independent test, CI or release acceptance. Reviewed: 2026-10-31.

close() now closes every sub-iterator and clears the list even when one
close() throws a RuntimeException or Error, then rethrows the first
failure with later ones suppressed. next() moves on to the next
sub-iterator itself and throws NoSuchElementException instead of
NullPointerException when nothing is left. After close(), hasNext() and
isValid() return false without probing any source, and a second close()
does nothing.

close apache#3444
@codecov

codecov Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 57.67%. Comparing base (608cb1d) to head (d8c2e42).
⚠️ Report is 27 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3445      +/-   ##
============================================
+ Coverage     57.27%   57.67%   +0.40%     
+ Complexity     6577     2589    -3988     
============================================
  Files           776      779       +3     
  Lines         68881    69161     +280     
  Branches       9431     9506      +75     
============================================
+ Hits          39449    39887     +438     
+ Misses        25804    25616     -188     
- Partials       3628     3658      +30     
Flag Coverage Δ *Carryforward flag
commons 96.37% <ø> (ø) Carriedforward from 608cb1d
hstore 36.19% <ø> (+0.10%) ⬆️
pd 52.80% <ø> (+0.64%) ⬆️ Carriedforward from 608cb1d
server_hbase 42.18% <ø> (+<0.01%) ⬆️ Carriedforward from 608cb1d
server_memory 55.99% <ø> (+0.01%) ⬆️ Carriedforward from 608cb1d
server_rocksdb 42.66% <ø> (+<0.01%) ⬆️ Carriedforward from 608cb1d
store 35.95% <ø> (+2.81%) ⬆️

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: no. Score: 9/10. Summary: close() closes every sub-iterator and clears the list even when one close() throws, next() advances through hasNext() and throws NoSuchElementException when exhausted or closed, and the five new FilterIteratorContractTest cases cover each path and pass in pd_store / store at this head.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Store MultiListIterator leaks sub-iterators on close failure, NPEs on bare next(), and probes sources after close

2 participants