[lxy-9602]

Purpose

Linked issue: #158

Improve real-time reads, spill behavior, and index support.

• Resolve historical file schemas for offset lookup, supporting compatible key renames and field reordering after recreating the context.
• Trim pending DV offsets per file using schema-aware statistics, with conservative fallback when statistics are unavailable.
• Compose input selection bitmaps with offset deletions, use ordered offset traversal, and allow sparse input bitmaps.
• Enable deduplicate spill through two-phase sealing: freeze data, key indexes, and DV checkpoints atomically, then spill outside the write mutex.
• Freeze without temporary IPC spill during PrepareCommit; retain spill behavior for explicit Seal.
• Include key indexes and distinct DV bitmaps in memory accounting, and release retired resources outside critical locks.
• Prepare committed snapshot refresh outside progress locks and reuse unchanged sidecar readers.
• Replace the bitmap-only restriction with exact BIGINT IN/NOT_IN capability validation, supporting bitmap, BSI, and range-bitmap.
• Reject non-main-branch real-time operations.

Tests

Added or updated coverage for:

• Historical schemas, offset-stat projection, and conservative fallback.
• Bitmap composition, sparse selections, and ordered offset filtering.
• Concurrent writes during spill, checkpoint stability, memory accounting, and resource cleanup.
• Snapshot refresh concurrency, failure/retry, and reader reuse.
• Embedded/external exact indexes and prepare-without-spill across append, deduplicate, PK MOR, and PK DV.

API and Format

• Add optional RealtimeStoreTwoPhaseSeal; custom stores retain the legacy path, while deduplicate spill requires this capability.
• Add FileIndexer::SupportsExactPredicate, defaulting to false. Binary plugins implementing FileIndexer require rebuilding.
• Keep capability validation at the configuration/schema level; existing-file consistency remains the application’s responsibility.

Documentation

Add a real-time guide covering supported modes, configuration, pluggable stores, read/commit lifecycle, indexes, spill, memory accounting, and limitations.

Generative AI tooling

Generated-by: OpenAI Codex