Skip to content

chore(ci): Claude issue triage and automatic security updates - #155

Merged
8bitAlex merged 4 commits into
mainfrom
ci/triage-and-security
Oct 30, 2026
Merged

8bitAlex merged 4 commits into
mainfrom
ci/triage-and-security

Conversation

@8bitAlex

Copy link
Copy Markdown
Owner

What this adds

Two groups of maintainer automation. Everything starts off: nothing merges until SECURITY_AUTOMERGE=true is set, and triage and the Claude fix lane need the Anthropic federation variables. No existing workflow is changed (claude.yml, build.yml, deploy.yml and src/resources/app.properties are untouched). Full description: docs/AUTOMATION.md.

Issue triage (triage.yml)

  • A new issue gets one comment (what happens, likely cause with path:line pointers, whether it is already fixed, duplicates, questions for the reporter) and the labels triaged, a category, priority: …, and needs-info or duplicate when they apply.
  • The repo is public, so issue text is hostile input. Claude has only Read/Grep/Glob/Write, no shell or network, and a read-only token; it writes one verdict file. A separate job with no Claude credentials validates it against a strict schema, sanitises it, and applies the labels and comment. It never closes, edits or assigns anything.
  • Issues by OWNER/MEMBER/COLLABORATOR are triaged automatically; an outsider's issue is triaged when someone with write access adds the triage label, or by workflow_dispatch. Widening this is a one-line change, documented.
  • Model: claude-sonnet-5-5, max 25 turns.

Security updates

  • Lane 1 (security-update.yml): a Dependabot PR that fixes a security alert is merged (squash) once the repo's CI is green on its exact head commit. Only manifests and lockfiles may change; no major bump of a direct dependency; at most 5 new indirect packages; registry packages only; every commit must be Dependabot's. It uses pull_request_target and never checks out or runs the PR's code.
  • Lane 2 (security-fix.yml): when CI fails on a bump (the new version needs a code change), Claude (claude-opus-5-5, 60 turns) fixes it. Claude cannot push: it produces a patch that a separate job checks (no .github/, allowlisted paths, size limit, version bump rule) and applies. Off unless SECURITY_CLAUDE_FIX=true.
  • Lane 3 (security-scan.yml): daily, finds fixable advisories with npm audit (site/) and govulncheck, tries the deterministic fix, builds the docs site and runs the Go tests, and opens one PR. Look-only on manual runs unless open_pr=true.
  • Dependency-only PRs need no version bump here (version-check only runs for src/**).

What you need to set after merging

  1. The Anthropic federation rule, then the variables ANTHROPIC_FEDERATION_RULE_ID and ANTHROPIC_ORGANIZATION_ID (service account / workspace ids only if the rule needs them). Claims the rule must accept (repo id 966580385, owner id 4384235):
    • aud https://api.anthropic.com, repository_owner 8bitAlex, repository 8bitAlex/raid
    • sub repo:8bitAlex/raid:ref:refs/heads/main (today; repo:8bitAlex@4384235/raid@966580385:ref:refs/heads/main if immutable subjects are enabled later)
    • job_workflow_ref …/triage.yml@refs/heads/main and …/security-fix.yml@refs/heads/main
  2. gh label create triage (the other new labels are created on first use).
  3. Settings > Actions > General > "Allow GitHub Actions to create and approve pull requests" (currently off; Lane 3 needs it to open a PR).
  4. gh variable set SECURITY_AUTOMERGE --body true to turn the merging on; SECURITY_CLAUDE_FIX=true for Lane 2.
  5. Optional secrets: SECURITY_ALERTS_TOKEN (fine-grained, Dependabot alerts: read, lets Lane 1 confirm an open alert) and SECURITY_BOT_TOKEN (lets pushes and PRs start workflows, in case the CodeQL ruleset rule waits on scanning results).

Testing

  • Hermetic script tests (fake gh, throwaway git repos): triage 364, claude-auth 21, security-policy 228, security 295 checks; bash -n, node --check and a YAML parse of every workflow are clean. A small new workflow, automation-tests.yml, runs them when .github/ changes; it passed on this branch.
  • Not run for real: the workflows themselves need this on main and the federation rule. After merging, try gh workflow run triage.yml -f issue=<n> on an existing issue, then gh workflow run security-scan.yml (look-only), and watch the next Dependabot security PR through Lane 1 with SECURITY_AUTOMERGE=true. Things to confirm on the first runs: the Dependabot-started pull_request_target run gets a write token, a workflow-created PR can merge past the CodeQL rule, and the required check names (build (ubuntu|macos|windows-latest), docs).
  • Not run locally: go build / go test (no Go changes) and the docs site build (no site/ changes).

Risks

Cost (triage is capped at 25 turns; Lane 2 is Opus, once per PR, off by default); prompt injection (no write path for Claude, strict validator); supply chain (a merge needs a published advisory fix, no major or new direct dependency, and green CI on the exact commit; a compromised release that is itself the fix and passes CI is the residual risk of any auto-merge).

🤖 Generated with Claude Code

8bitAlex and others added 4 commits October 29, 2026 12:45
…ge label)

Port of the cp-slider triage: prepare (read-only, who may start it) ->
Claude with Read/Grep/Glob/Write only and no network, writing one verdict
file -> apply, which validates the verdict against a strict schema, makes
the text read-only and posts one comment plus labels.

raid is public, so the default is conservative: automatic only for issues
opened by an OWNER/MEMBER/COLLABORATOR with write access; an outsider's
issue is triaged when a maintainer adds the `triage` label; bots and
issues labelled triaged/dependencies/security are skipped. One run per
issue. Federation auth, claude-sonnet-5-5.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ix, daily scan)

Three lanes, all behind the repository variable SECURITY_AUTOMERGE (off by
default; Claude's lane also needs SECURITY_CLAUDE_FIX):

1. security-update.yml (pull_request_target, never checks out the PR): a
   Dependabot PR that fixes a published advisory (open alert, or OSV), no
   semver-major bump of a direct dependency, only the four manifests, all
   commits Dependabot's, manifest checks against the merge base; waits for the
   exact commit's CI and squash-merges with --match-head-commit.
2. security-fix.yml (workflow_dispatch): Claude (claude-opus-5-5, 60 turns,
   go build/vet/test only) adapts the code once per PR; a separate job checks
   the patch (no .github/, no manifests, size, version bump), pushes, reruns
   CI and the same assessment, then merges. Otherwise a comment and one
   `security` issue.
3. security-scan.yml (daily): npm audit / govulncheck, deterministic fixes
   without breaking changes, docs build and go test must pass, one PR per run.

Federation auth for Claude, patches treated as data, per-job permissions.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…n .github/ changes

Bash and node tests with a fake gh (backed by a throwaway git repository),
curl, npm, go and govulncheck. A small separate workflow runs them (plus bash
-n, node --check and a YAML parse) only for changes under .github/, so the
main build matrix is untouched.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…redentials)

What each workflow does, the variables and secrets, the OIDC claims the
Anthropic federation rule must accept, labels, how to run and stop things by
hand, the public-repository policy for triage, and how to verify on main.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.02%. Comparing base (f5afadf) to head (37c50de).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #155   +/-   ##
=======================================
  Coverage   93.02%   93.02%           
=======================================
  Files          48       48           
  Lines        4487     4487           
=======================================
  Hits         4174     4174           
  Misses        201      201           
  Partials      112      112           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@8bitAlex
8bitAlex merged commit 6824db6 into main Oct 30, 2026
15 checks passed
@8bitAlex
8bitAlex deleted the ci/triage-and-security branch October 30, 2026 02:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant