Skip to content

feat(cache): restore symlinks a task creates - #918

Open
wan9chi wants to merge 2 commits into
feat/cache-empty-foldersfrom
feat/cache-created-symlinks
Open

wan9chi wants to merge 2 commits into
feat/cache-empty-foldersfrom
feat/cache-created-symlinks

Conversation

@wan9chi

@wan9chi wan9chi commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Motivation

A task that creates a symbolic link doesn't have it among its outputs unless an output glob matches it. Nitro's dependency tracer, for example, links .output/server/node_modules/tslib into .nf3. fspy records nothing for symlink, so a hit leaves the link out. When the task checks the path first, as nf3 does, the link is a plain input and a clean checkout misses.

Changes

fspy records the path of a new link as a write, before the call:

  • Unix preload: symlink and symlinkat.
  • Linux seccomp supervisor: the same calls for static binaries.
  • Windows: NtFsControlFile with FSCTL_SET_REPARSE_POINT(_EX), which covers CreateSymbolicLinkW and junctions.

vt needs no change: a written symlink counts as a file, and the archive stores links as links (#885).

E2E cases: symlink_in_output (synthetic, vtt symlink) and nitro_auto_outputs_restore_links (the nitro_build_cache Nitro build with automatic tracking only). Both run only on Unix in the first commit, because Windows already behaves differently there, and on every platform after.

Part of a stack that lets automatic tracking cache tasks which read their own outputs and list the folders holding them, so they also hit on a clean checkout.

🤖 Generated with Claude Code


Note

Medium Risk
Changes syscall/NT detour behavior for symlink creation across all tracked task runs; incorrect recording could affect cache fingerprints and restored outputs, but the pattern matches existing hard-link and rename tracking.

Overview
Symbolic links a task creates are now tracked as outputs and restored on cache hit, even when no output glob covers them—fixing cases like Nitro’s dependency links under .output/server/node_modules and clean-checkout misses when the task stat’d the link path first.

fspy records the link path as a write before the syscall returns (failed creates still count): symlink / symlinkat on Unix preload and Linux seccomp; on Windows, NtFsControlFile when FSCTL_SET_REPARSE_POINT(_EX) runs (covers CreateSymbolicLinkW and junctions). The symlink target string is not accessed for tracking.

Vite Task logic is unchanged—written symlinks already archive and replay as links (#885); rebuildable docs note Windows junctions count as written symlinks, not folders. vtt symlink works on Windows for e2e. New snapshot tests: symlink_in_output and nitro_auto_outputs_restore_links (auto tracking only).

Reviewed by Cursor Bugbot for commit f7f77b5. Bugbot is set up for automated code reviews on this repo. Configure here.

@wan9chi
wan9chi added this pull request to stack #920 October 30, 2026 03:17
@wan9chi
wan9chi force-pushed the feat/cache-created-symlinks branch from ce2ff95 to 6f295b5 Compare October 30, 2026 03:19
@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

fspy benchmark

linux

dynamic/launch             change  -0.55%  [ -5.12% ..  +3.81%]  overhead   +60.84%
dynamic/access             change  +0.42%  [ -3.47% ..  +7.41%]  overhead   +27.85%
dynamic/access-relative    change  +1.78%  [ -1.78% ..  +5.03%]  overhead   +79.65%
dynamic/access-contended   change  +0.32%  [ -5.64% ..  +5.43%]  overhead   +53.92%
static/launch              change  +0.34%  [ -3.92% ..  +3.83%]  overhead  +197.70%
static/access              change  -0.16%  [ -1.99% ..  +1.32%]  overhead +1561.95%
static/access-relative     change  -0.02%  [ -1.23% ..  +1.42%]  overhead +2055.79%
static/access-contended    change  +0.26%  [ -0.41% ..  +0.70%]  overhead +2099.90%

macos

dynamic/launch             change  +0.02%  [ -4.93% ..  +5.94%]  overhead  +241.34%
dynamic/access             change  +0.65%  [ -5.25% ..  +9.49%]  overhead    +4.38%
dynamic/access-relative    change  -3.16%  [-10.93% ..  +3.49%]  overhead  +224.93%
dynamic/access-contended   change  +2.06%  [-13.13% .. +20.40%]  overhead    +2.45%

windows

dynamic/launch             change  +0.41%  [ -2.90% ..  +2.46%]  overhead   +25.51%
dynamic/access             change  +0.39%  [ -0.20% ..  +1.19%]  overhead    +1.18%
dynamic/access-relative    change  +0.00%  [ -0.75% ..  +0.78%]  overhead    +1.17%
dynamic/access-contended   change  +0.19%  [ -0.92% ..  +1.28%]  overhead    +1.31%

@wan9chi wan9chi added the ci: full Run the slow CI jobs (macOS x64, musl) on this pull request label Oct 30, 2026
@wan9chi
wan9chi force-pushed the feat/cache-created-symlinks branch from 6f295b5 to 50eb6eb Compare October 30, 2026 04:07
@wan9chi wan9chi added ci: full Run the slow CI jobs (macOS x64, musl) on this pull request and removed ci: full Run the slow CI jobs (macOS x64, musl) on this pull request labels Oct 30, 2026
@wan9chi
wan9chi force-pushed the feat/cache-created-symlinks branch from 50eb6eb to 3265412 Compare October 30, 2026 06:31
@wan9chi
wan9chi force-pushed the feat/cache-created-symlinks branch from 3265412 to 25a585d Compare October 30, 2026 07:08
wan9chi and others added 2 commits November 1, 2026 20:51
Two cases for a task that creates symbolic links. In `symlink_in_output`,
the task links `out/link` to `out/target.txt` with `vtt symlink`, which now
works on Windows too, making a directory link when the target is a
directory. In `nitro_auto_outputs_restore_links`, the Nitro build of
`nitro_build_cache` runs with automatic tracking only, without `input` or
`output` globs; nf3 links `tslib` and each package's `tslib` into `.nf3`.

fspy records nothing for `symlink` on Unix, so a link a task creates isn't
an output. Removing `out/link` is a cache hit that leaves it missing. nf3
checks each path before linking it, so Nitro's links are plain inputs,
and removing `.output`, as on a clean checkout, is a cache miss. Several
inputs are then missing, and vt names whichever its parallel validation
finds first, so the step redacts its miss reason.

Both cases are Unix only for now. On Windows, `CreateSymbolicLinkW` opens
the new link without write access, so fspy records it as a read, and
removing `out/link` is a cache miss. nf3 makes junctions there, which
libuv opens with write access before setting the reparse point, so
they're recorded as written already.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A task that creates a symbolic link, like Nitro's dependency tracer
linking `.output/server/node_modules/tslib` into `.nf3`, didn't have the
link among its outputs unless an `output` glob matched it: fspy recorded
nothing for `symlink`, so a hit left the link out, and when the task
checked the path first, as nf3 does, the link was a plain input, so
removing it, as on a clean checkout, was a cache miss.

fspy now records the path of a new link as a write, before the call, so a
failed call is recorded too. The target is stored as text, not accessed,
and needs no record.

- Unix preload: `symlink` and `symlinkat`.
- Linux seccomp supervisor, for static binaries: `symlinkat`, plus
  `symlink` on x86_64.
- Windows: `NtFsControlFile` with `FSCTL_SET_REPARSE_POINT(_EX)`, for the
  file the reparse point is set on. `CreateSymbolicLinkW` creates the link
  with an open that lacks write access, recorded as a read, and then sets
  its reparse point. A junction, which libuv makes for Node's `fs.symlink`
  with `'junction'` and nf3 uses on Windows, is a folder with a mount
  point set on it; libuv opens the folder with write access first, so
  those junctions were recorded as written already.

vt needs no change: `Rebuildable` treats a written symlink as a file, and
the archive stores links as links (#885). std reports a junction as a
symlink, not a folder, so the mkdir recorded for it doesn't make it an
empty folder, and the write makes it rebuildable. Both e2e cases now run
on Windows too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@wan9chi
wan9chi force-pushed the feat/cache-created-symlinks branch from 25a585d to f7f77b5 Compare November 1, 2026 11:55
@wan9chi wan9chi added ci: full Run the slow CI jobs (macOS x64, musl) on this pull request and removed ci: full Run the slow CI jobs (macOS x64, musl) on this pull request labels Nov 1, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci: full Run the slow CI jobs (macOS x64, musl) on this pull request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant