Skip to content

Recover long-context requests rejected by the model window - #157

Merged
mrmps merged 1 commit into
mainfrom
codex/jev-full-input-context-fallback
Oct 19, 2026
Merged

mrmps merged 1 commit into
mainfrom
codex/jev-full-input-context-fallback

Conversation

@mrmps

@mrmps mrmps commented Oct 18, 2026 •

Copy link
Copy Markdown
Owner

Some full documents pass the conservative local estimate but exceed Jev’s actual tokenizer limit, turning otherwise valid long-context requests into503 errors. An explicit provider context refusal now resumes the existing screening path; other inference failures still fail.

Successful direct reads alone increment full_input_passes. Rejected attempts preserve truthful usage, and recovery settles the original document tariff once. The exact previously failing creator request returned all9 ratings and4 flags through the built Worker against real TypeSafe.

No UI changes; visual before/after evidence is not applicable.

Summary by CodeRabbit

  • Bug Fixes
    • When a provider rejects an oversized document, processing can now recover by screening the complete document and selecting relevant evidence instead of failing.
    • Recovered requests retain pricing based on the original document, including for single and dimensions requests.

@coderabbitai

coderabbitai Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 666b4fc6-ad34-4950-8bc9-be797fa2749a
📥 Commits

Reviewing files that changed from the base of the PR and between a823bb4 and 3659388.

📒 Files selected for processing: 3
  • e2e/long-context-billing.ts
  • e2e/long-context.mjs
  • src/long-context.ts

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 0 remain after this review.

📝 Walkthrough

The long-context classifier now falls back to chunked classification when a full-input request receives a max_tokens_exceeded error. E2E scenarios verify recovery for single, dimensions, and billing requests, including screening behavior and original-document pricing.

Changes: Long-context recovery

Layer / File(s) Summary
Handle max-token rejections
src/long-context.ts
The phase preserves token usage for max-token rejections. Full-input classification continues with chunked classification for that error and rethrows other errors.
Verify recovery and billing
e2e/long-context.mjs, e2e/long-context-billing.ts
Provider fixtures can reject qualifying full-input requests. E2E scenarios verify recovery through screening, the absence of full-input passes, and pricing based on the original document.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~15 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 36593

Long-context requests that the provider rejects for exceeding the model window now recover through the existing chunked screening path. Other failures behave as before. No concrete merge-blocking risk was found.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 3 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: recovering long-context requests that the model rejects because of its context window.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
  • Fix all pre-merge checks with AI
✨ Finishing Touches: 💡 1
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@mrmps
mrmps marked this pull request as ready for review October 18, 2026 23:59
@mrmps
mrmps merged commit b0e3b44 into main Oct 19, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant