You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Recover long-context requests rejected by the model window - #157
Some full documents pass the conservative local estimate but exceed Jev’s actual tokenizer limit, turning otherwise valid long-context requests into503 errors. An explicit provider context refusal now resumes the existing screening path; other inference failures still fail.
Successful direct reads alone increment full_input_passes. Rejected attempts preserve truthful usage, and recovery settles the original document tariff once. The exact previously failing creator request returned all9 ratings and4 flags through the built Worker against real TypeSafe.
No UI changes; visual before/after evidence is not applicable.
Summary by CodeRabbit
Bug Fixes
When a provider rejects an oversized document, processing can now recover by screening the complete document and selecting relevant evidence instead of failing.
Recovered requests retain pricing based on the original document, including for single and dimensions requests.
No actionable comments were generated in the recent review. 🎉
ℹ️ Recent review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Advanced
Run ID: 666b4fc6-ad34-4950-8bc9-be797fa2749a
📥 Commits
Reviewing files that changed from the base of the PR and between a823bb4 and 3659388.
📒 Files selected for processing: 3
e2e/long-context-billing.ts
e2e/long-context.mjs
src/long-context.ts
Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 0 remain after this review.
📝 Walkthrough
The long-context classifier now falls back to chunked classification when a full-input request receives a max_tokens_exceeded error. E2E scenarios verify recovery for single, dimensions, and billing requests, including screening behavior and original-document pricing.
Changes: Long-context recovery
Layer / File(s)
Summary
Handle max-token rejections src/long-context.ts
The phase preserves token usage for max-token rejections. Full-input classification continues with chunked classification for that error and rethrows other errors.
Verify recovery and billing e2e/long-context.mjs, e2e/long-context-billing.ts
Provider fixtures can reject qualifying full-input requests. E2E scenarios verify recovery through screening, the absence of full-input passes, and pricing based on the original document.
Long-context requests that the provider rejects for exceeding the model window now recover through the existing chunked screening path. Other failures behave as before. No concrete merge-blocking risk was found.
Pre-merge checks: 4 · 1
Failed checks: 1 warning
Check name
Status
Explanation
Resolution
Docstring Coverage
Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 3 files.
Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed checks: 4 passed
Check name
Status
Explanation
Description Check
Check skipped - CodeRabbit’s high-level summary is enabled.
Title check
The title clearly and concisely describes the main change: recovering long-context requests that the model rejects because of its context window.
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Fix all pre-merge checks with AI
✨ Finishing Touches: 💡 1
📝 Generate docstrings: 💡
Commit to this branch
Create a new PR
🧪 Generate unit tests (beta)
Commit to this branch
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Some full documents pass the conservative local estimate but exceed Jev’s actual tokenizer limit, turning otherwise valid long-context requests into503 errors. An explicit provider context refusal now resumes the existing screening path; other inference failures still fail.
Successful direct reads alone increment
full_input_passes. Rejected attempts preserve truthful usage, and recovery settles the original document tariff once. The exact previously failing creator request returned all9 ratings and4 flags through the built Worker against real TypeSafe.No UI changes; visual before/after evidence is not applicable.
Summary by CodeRabbit