Skip to content

feat(cache): let a hit rebuild empty folders the task made - #917

Open
wan9chi wants to merge 2 commits into
feat/cache-rename-targetsfrom
feat/cache-empty-folders
Open

wan9chi wants to merge 2 commits into
feat/cache-rename-targetsfrom
feat/cache-empty-folders

Conversation

@wan9chi

@wan9chi wan9chi commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Motivation

A task that makes a folder and leaves it empty can't be rebuilt by a hit today; astro build, for example, makes .astro/collections. fspy records nothing for mkdir, and the archive holds only files, so the folder holding the empty one stays pinned and a clean checkout misses.

Changes

  • fspy records mkdir as a new access kind, Mkdir, before the call. This covers the Unix preload, the Linux seccomp supervisor and Windows NtCreateFile creating a directory; on Windows those calls were recorded as reads.
  • A mkdir'd folder that is empty after the run is rebuildable and a starting point for the climb.
  • The archive gets directory entries for those folders, filtered by output exclusions like files, and a restore recreates them.

E2E cases: empty_folder_in_output (synthetic) and astro_empty_collections (astro build with Astro 6.4.8, added to packages/tools with a scoped vite override; true clean checkout).

Part of a stack that lets automatic tracking cache tasks which read their own outputs and list the folders holding them, so they also hit on a clean checkout.

🤖 Generated with Claude Code


Note

Medium Risk
Changes cross-platform fspy interception and cache fingerprint/archive replay logic; incorrect tracking would cause false hits or misses, though the PR adds substantial unit and e2e tests.

Overview
Automatic file tracking now records paths created without a normal open: Mkdir for directory creation, RenameTarget for rename destinations, and hard-link destinations as writes. This is wired through Unix seccomp/preload, Windows NtCreateFile / NtSetInformationFile, and new IPC PathAccess variants (recorded even when the syscall fails).

Task cache builds on those signals in Rebuildable: empty post-run mkdir folders are outputs and skip input fingerprinting; folder renames expand to the files underneath for archiving and “may be missing” validation; gone temp dirs the task wrote in are treated like rebuildable outputs. Output tar.zst archives now include explicit empty directory entries, and cache restore recreates them.

E2E coverage adds empty_folder_in_output and astro_empty_collections (Astro 6 in dev deps); existing rename/Vitest snapshot fixtures now expect cache hits after deleting outputs.

Reviewed by Cursor Bugbot for commit e038c4c. Bugbot is set up for automated code reviews on this repo. Configure here.

@socket-security

socket-security Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addednpm/​astro@​6.4.897258898100

View full report

@socket-security

socket-security Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Warning

Review the following alerts detected in dependencies.

According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.

Priority Alert  (click "▶" to expand/collapse) Action
Critical priority
Critical CVE: Astro: Remote code execution through AVIF image optimization

CVE: GHSA-26w7-cxv4-gfx2 Astro: Remote code execution through AVIF image optimization (CRITICAL)

Affected versions: < 7.2.8

Patched version: 7.2.8

From: packages/tools/package.json → npm/astro@6.4.8

ℹ Read more on: This package | This alert | What is a critical CVE?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Remove or replace dependencies that include known critical CVEs. Consumers can use dependency overrides or npm audit fix --force to remove vulnerable dependencies.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/astro@6.4.8. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn

View full report

@wan9chi
wan9chi added this pull request to stack #920 October 30, 2026 03:17
@wan9chi
wan9chi force-pushed the feat/cache-empty-folders branch from d7c8a30 to 78b8ee8 Compare October 30, 2026 03:20
@wan9chi wan9chi added the ci: full Run the slow CI jobs (macOS x64, musl) on this pull request label Oct 30, 2026
@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

fspy benchmark

linux

dynamic/launch             change  +0.42%  [ -3.17% ..  +4.61%]  overhead   +59.74%
dynamic/access             change  -0.44%  [ -5.25% ..  +4.30%]  overhead   +19.90%
dynamic/access-relative    change  +0.00%  [ -3.60% ..  +4.10%]  overhead   +67.02%
dynamic/access-contended   change  -0.80%  [ -3.03% ..  +6.07%]  overhead   +34.87%
static/launch              change  +1.83%  [ -3.16% ..  +7.45%]  overhead  +193.92%
static/access              change  -0.39%  [ -4.45% ..  +2.69%]  overhead +1434.27%
static/access-relative     change  -0.75%  [ -3.11% ..  +1.29%]  overhead +1936.22%
static/access-contended    change  -0.07%  [ -2.09% ..  +1.73%]  overhead +2103.13%

macos

dynamic/launch             change  +0.61%  [ -4.28% ..  +4.74%]  overhead  +233.14%
dynamic/access             change  -0.68%  [ -6.64% ..  +6.23%]  overhead    -0.38%
dynamic/access-relative    change  +2.67%  [ -2.83% ..  +9.18%]  overhead  +264.92%
dynamic/access-contended   change  -2.09%  [ -7.51% ..  +5.53%]  overhead    +6.00%

windows

dynamic/launch             change  -0.37%  [ -9.78% ..  +7.10%]  overhead   +23.39%
dynamic/access             change  +0.19%  [ -1.09% ..  +0.93%]  overhead    +1.68%
dynamic/access-relative    change  +0.18%  [ -1.47% ..  +0.94%]  overhead    +0.94%
dynamic/access-contended   change  +0.18%  [ -1.07% ..  +1.24%]  overhead    +1.80%

@wan9chi
wan9chi force-pushed the feat/cache-empty-folders branch from 78b8ee8 to 922a074 Compare October 30, 2026 04:07
@wan9chi wan9chi added ci: full Run the slow CI jobs (macOS x64, musl) on this pull request and removed ci: full Run the slow CI jobs (macOS x64, musl) on this pull request labels Oct 30, 2026
@wan9chi
wan9chi force-pushed the feat/cache-empty-folders branch from 922a074 to 2166c58 Compare October 30, 2026 06:31
@wan9chi
wan9chi force-pushed the feat/cache-empty-folders branch from 2166c58 to b81c7bb Compare October 30, 2026 07:08

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit b81c7bb. Configure here.

Comment thread crates/vt/src/session/cache/archive.rs Resolved
wan9chi and others added 2 commits November 1, 2026 20:50
Two cases for a task whose output folder holds an empty folder it made.
In `empty_folder_in_output`, the task makes `out/empty`, writes
`out/a.txt` and stats `out`, through `vtt fs`. In
`astro_empty_collections`, `astro build` (Astro 6, added to
`packages/tools`) on a one-page site makes `.astro/collections` and leaves
it empty, next to the `.d.ts` files it writes in `.astro`, and stats both
folders.

Today a folder holding an empty folder isn't rebuildable, since mkdir
isn't recorded, so removing it, as on a clean checkout, is a cache miss.

Astro needs Vite 7, so a scoped `astro>vite` override keeps the global
`vite` override from handing it the Vite+ fork. esbuild and sharp, which
Astro depends on, get their prebuilt binaries as optional dependencies, so
their build scripts are skipped. The Astro case removes `.astro`, `dist`,
`node_modules/.astro` and `node_modules/.vite/deps`, not all of
`node_modules/.vite`, which holds the task cache too, and recreates
nothing, as on a clean checkout. Several inputs are then missing, and vt
names whichever its parallel validation finds first, so the next step
redacts its miss reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A task that makes a folder and leaves it empty, like `astro build` making
`.astro/collections`, pinned the folder holding it as an input: fspy
recorded nothing for a mkdir, so the empty folder wasn't rebuildable, and
the archive held only files, so a hit couldn't recreate it. Removing the
folder, as on a clean checkout, was a cache miss.

fspy now records the path of a mkdir as a mkdir, a new access kind, before
the call, so a failed call, like one finding the folder there already, is
recorded too:

- Unix preload: `mkdir` and `mkdirat`.
- Linux seccomp supervisor, for static binaries: `mkdirat`, plus `mkdir`
  on x86_64.
- Windows: `NtCreateFile` with `FILE_DIRECTORY_FILE` and `FILE_CREATE` or
  `FILE_OPEN_IF`, where `CreateDirectoryW` ends. These opens were recorded
  as reads; a mkdir isn't one on Unix either.

`Rebuildable` takes the mkdirs that are empty folders after the run
(`.DS_Store` aside) as rebuildable folders and climbs from them too, so a
folder holding only such folders, rebuildable folders and files the task
wrote is rebuildable. The archive gets directory entries for the empty
folders, filtered by output exclusions and `ignoreOutput` like files, and
restoring recreates them. A folder holding anything needs no entry, since
what it holds recreates it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@wan9chi
wan9chi force-pushed the feat/cache-empty-folders branch from b81c7bb to e038c4c Compare November 1, 2026 11:55
@wan9chi wan9chi added ci: full Run the slow CI jobs (macOS x64, musl) on this pull request and removed ci: full Run the slow CI jobs (macOS x64, musl) on this pull request labels Nov 1, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci: full Run the slow CI jobs (macOS x64, musl) on this pull request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant