Skip to content

feat(cache): let a listed folder lack entries a hit can rebuild - #915

Open
wan9chi wants to merge 2 commits into
feat/cache-rebuildable-foldersfrom
feat/cache-listing-ignored-entries
Open

wan9chi wants to merge 2 commits into
feat/cache-rebuildable-foldersfrom
feat/cache-listing-ignored-entries

Conversation

@wan9chi

@wan9chi wan9chi commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Motivation

A task that lists a folder holding its own outputs pins those outputs as entries of the listing. TypeScript 7's tsc with no include, for example, lists the package root that holds its lib output folder. A clean checkout lacks them, so it misses. Today only an entry named dist is spared, by a hard-coded skip.

Changes

When a listing is fingerprinted, the entries the task can rebuild are left out and their names recorded as ignored_entries: a file or symlink it wrote, or a rebuildable folder. Validation drops the same names from the current listing before comparing, so they may be missing or present. Any other entry that differs still misses and is named. The cache schema version is bumped.

E2E cases: list_folder_holding_output (synthetic; an unrelated new file still misses) and typescript_lib_out_dir (TypeScript 7, outDir: "lib").

Part of a stack that lets automatic tracking cache tasks which read their own outputs and list the folders holding them, so they also hit on a clean checkout.

🤖 Generated with Claude Code


Note

Medium Risk
Changes post-run fingerprint serialization and validation semantics for directory listings, with a schema bump that invalidates existing caches; incorrect ignored-entry logic could cause stale hits or spurious misses.

Overview
When automatic tracking records a directory listing as a cache input, entries the task can rebuild (files it wrote or output-only folders) are no longer part of the stored listing fingerprint. Their names are saved in new InferredInput.ignored_entries, and validation strips the same names from the live listing before comparison—so missing lib, gen/, or similar output after a clean checkout can still cache hit and restore outputs.

Rebuildable gains is_rebuildable_entry for per-entry checks; CACHE_SCHEMA_VERSION is bumped to 21 (fresh cache). Docs and changelog describe the behavior; e2e fixtures cover a synthetic list-and-write task and TypeScript tsc with outDir: "lib". The typescript_incremental snapshot no longer needs a Unix-only cfg now that listing semantics match Windows too.

Reviewed by Cursor Bugbot for commit 912534a. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

fspy benchmark

linux

dynamic/launch             change  +1.09%  [ -4.29% ..  +6.84%]  overhead   +56.12%
dynamic/access             change  -1.01%  [-32.16% .. +19.76%]  overhead   +24.93%
dynamic/access-relative    change  +0.85%  [ -4.48% .. +12.50%]  overhead   +72.48%
dynamic/access-contended   change  -1.94%  [-11.45% ..  +5.69%]  overhead   +11.34%
static/launch              change  +0.30%  [ -4.46% ..  +7.01%]  overhead  +189.42%
static/access              change  -0.21%  [ -6.12% ..  +3.00%]  overhead +1314.25%
static/access-relative     change  -0.67%  [-14.84% ..  +3.90%]  overhead +1928.07%
static/access-contended    change  -0.01%  [ -9.80% ..  +8.22%]  overhead +1267.48%

macos

dynamic/launch             change  -0.50%  [ -4.96% ..  +5.11%]  overhead  +249.05%
dynamic/access             change  -0.26%  [ -6.58% .. +11.86%]  overhead    +5.21%
dynamic/access-relative    change  +1.08%  [ -7.47% ..  +8.32%]  overhead  +257.10%
dynamic/access-contended   change  -0.22%  [ -9.86% ..  +5.37%]  overhead    +0.78%

windows

dynamic/launch             change  -0.33%  [ -2.59% ..  +1.52%]  overhead   +25.42%
dynamic/access             change  +0.00%  [ -0.55% ..  +0.92%]  overhead    +1.13%
dynamic/access-relative    change  +0.18%  [ -0.73% ..  +0.92%]  overhead    +0.92%
dynamic/access-contended   change  +0.00%  [ -0.88% ..  +0.54%]  overhead    +1.25%

@wan9chi
wan9chi force-pushed the feat/cache-listing-ignored-entries branch from 531c594 to 524114b Compare October 30, 2026 04:07
@wan9chi wan9chi added the ci: full Run the slow CI jobs (macOS x64, musl) on this pull request label Oct 30, 2026
@wan9chi
wan9chi force-pushed the feat/cache-listing-ignored-entries branch from 524114b to f92e4fd Compare October 30, 2026 06:31
…akes it miss

Two cases for a task that lists a folder holding its own output folder. In
`list_folder_holding_output`, the task makes `gen`, writes `gen/a.txt` and
lists the package root, through `vtt fs`. In `typescript_lib_out_dir`,
TypeScript 7's `tsc` with no `include` compiles `src` into `lib`: it lists
the package root and stats `lib`.

Today removing the output folder, as on a clean checkout, is a cache miss
naming the folder as removed from the listing, although the cached outputs
hold everything in it. A new entry in the listing is a miss too, and must
stay one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A task that lists a folder holding its own outputs, like TypeScript 7's
`tsc` with no `include` listing the package root that holds its `lib`
output folder, pinned those outputs as entries of the listing. Removing
them, as on a clean checkout, was a cache miss although a hit restores
them.

When a listing is fingerprinted, the entries the task can rebuild, a file
or symlink it wrote or a folder holding only what it can rebuild, are now
left out, and their names are recorded as `ignored_entries` on the
inferred input. Validation removes the same names from the current
listing before comparing, so they may be missing or present, while any
other entry that differs still misses and is named in the message.
`Rebuildable` answers whether an entry is rebuildable from the written
paths and the folders its walk already found, so no folder is listed
again. `.DS_Store` and `dist` stay skipped in every listing and are never
recorded.

The cache schema version is bumped for the new inferred input format.

`typescript_incremental` now runs on Windows too, where `tsc` also lists
the package root, which holds the `tsconfig.tsbuildinfo` it wrote.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@wan9chi
wan9chi force-pushed the feat/cache-listing-ignored-entries branch from f92e4fd to 912534a Compare October 30, 2026 07:08

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci: full Run the slow CI jobs (macOS x64, musl) on this pull request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant