You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit f2ad92f
Browse filesBrowse the repository at this point in the historyBrowse files
feat(heavy): let long jobs yield, go round by worktree, and backfill
A long job held the solo lock until it ended: a 200 run balance batch, a
nightly, or a three profile splash run kept every other agent waiting,
and one worktree's back-to-back jobs could hog the queue. A waiting
Timing job also made every later Shared job wait behind it, even when
the Shared job fit the budget beside the running CPU job.
- Long jobs yield. The guard gives its job HEAVY_TURN_DIR, and the job
calls yieldTurn (tools/heavy/turn.ts) at each boundary. When another
job waits, the guard lets go of its file, queues again as a new
arrival, and grants the job the WORKERS of its next turn; when nobody
waits it goes on at once. The balance bot plays chunks of about 25
runs (rounded to whole rounds of its workers, HEAVY_CHUNK_RUNS to set
or 0 to turn off), a wave pass goes by snapshot slot, the nightly
yields between its jobs, splash.ts between profiles, and perf-check
between scenarios. Runs stay seeded by index, campaign chunks merge in
seed order and wave runs are sorted, so the report matches an
unchunked batch. A requeued batch is not counted or refused again and
its second wait never gives up.
- The queue goes round by worktree. A guard records when its worktree's
turn ended (held-<key>.json); a worktree with a turn in the last 5
minutes goes behind other worktrees' waiting jobs, then arrival order.
- Backfill. A Shared job that fits may start ahead of a Timing or CPU
waiter that only waits for a running CPU job, for up to 2 minutes of
that waiter's wait, never when the waiter waits for Shared jobs to
drain.
- Leases. A solo turn is a 10 minute lease (HEAVY_LEASE_MINUTES) in the
lock file. Past it the job keeps running, but its guard says so on
stderr and waiting jobs name it; a kept turn at a boundary renews it.
decision.test.ts covers the queue order, round robin, handover, leases,
backfill, and a model of the queue over time: no job waits more than 15
minutes behind an hour of chunked runs (an unchunked hour breaks it),
and in a typical mix screenshot jobs wait 6 s on average and 70 s at
most (160 s without backfill). A scratch run with HEAVY_STATE_DIR showed
two Shared jobs from two directories start at a fake CPU job's next
chunk boundary, a backfilled shot start beside it, and the lease
warnings.
Copy file name to clipboardExpand all lines: AGENTS.md
+4-1Lines changed: 4 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -41,7 +41,10 @@ Before you finish a change, run `typecheck`, `lint`, `test`, and `build`. All fo
41
41
42
42
- Run every heavy job through the guard: `bun run balance`, `bun run perf:check`, or `bun run heavy -- <command...>` for anything else heavy, such as an audit or a shots script. Never call `tools/balance/simulate.ts` or a heavy audit directly.
43
43
- The guard aims at about 70% of the machine: a 1 minute load near 0.70 x cores (8.4 on 12, `HEAVY_TARGET_LOAD` to change it). A job starts only when the load, the jobs started in the last minute, and its own cost fit under that target.
44
-
-**Timing** jobs (`perf:check` and the timing audits, `splash.ts`, and any command it does not know) run alone. **CPU** jobs (the balance bot, nightly, wave snapshots, determinism) run one at a time beside audits and size their `WORKERS` to the budget, up to 8. **Shared** jobs (the shots scripts, `taps.ts`, and the other screenshot and functional audits) run up to 4 at once, each costing about one core per browser context. Jobs queue fairly by arrival. `--timing`, `--cpu`, or `--shared` before the command overrides the class.
44
+
-**Timing** jobs (`perf:check` and the timing audits, `splash.ts`, and any command it does not know) run alone. **CPU** jobs (the balance bot, nightly, wave snapshots, determinism) run one at a time beside audits and size their `WORKERS` to the budget, up to 8. **Shared** jobs (the shots scripts, `taps.ts`, and the other screenshot and functional audits) run up to 4 at once, each costing about one core per browser context. `--timing`, `--cpu`, or `--shared` before the command overrides the class.
45
+
-**Long jobs yield.** The balance bot plays its runs in chunks of about 25 (a wave pass by snapshot slot), the nightly yields between its jobs, `splash.ts` between profiles, and `perf:check` between scenarios. At each boundary the guard hands over to any waiting job and queues again, so nobody waits long behind one batch. Results are the same as one unchunked run. A new long script should do the same with `yieldTurn` from `tools/heavy/turn.ts`.
46
+
-**The queue goes round by worktree.** A worktree that just had a turn goes behind other worktrees' waiting jobs; otherwise jobs go by arrival. A Shared job may start ahead of a Timing or CPU job that only waits for a running CPU job (backfill).
47
+
-**A solo turn is a 10 minute lease** (`HEAVY_LEASE_MINUTES`). A job past it keeps running, but the guard says so loudly and waiting jobs name it: split such a job so it yields.
45
48
-`bun run test` sizes vitest's workers to what is left of the budget (at least 2) and counts itself, so test runs from several agents share the target. CI is unaffected.
46
49
- The guard refuses a sixth balance batch per branch without `--allow-more` (`bun run heavy -- --reset-budget` starts the count again). How it works is in [`tools/heavy/README.md`](./tools/heavy/README.md).
47
50
- Cap tuning at about three batches per change. Use 100 runs to sort ideas and 200 only to confirm the final numbers. Results within a few points of a target are fine: list misses as follow-ups instead of running more batches.
0 commit comments