Skip to content

feat(rules): run SQL Rules through the SQLClean engine - #1283

Merged
kvlonge merged 9 commits into
mainfrom
feat/sqlclean-engine
Nov 1, 2026
Merged

kvlonge merged 9 commits into
mainfrom
feat/sqlclean-engine

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Why

SQL Rules now live in the standalone SQLClean engine. SQLBuild runs them through it, so both tools share one rules system: an omitted select runs the default Rules, select = [] turns configurable Rules off, and compiler validation stays on.

Changes

  • Run SQL Rules through the sqlclean crate, pinned to the v0.4.2 release commit. Codes are now SQLCLEAN<nnn>.
  • Default-Rules notice: once, human output only.
  • SQL Rules expand with the compiled --target/--vars on every compile path. Audit and test files with sql_analysis false are skipped.
  • format --fix keeps its eight-code eligibility, verification and exit semantics.
  • Fixtures: incidental SQL rewritten; where the SQL shape is the subject, a scoped rule_ignores gives the reason.
  • Perf guards run default Rules at every size. One select = [] guard remains: the 3000-model cache twin, in the same job, at the old budgets (cold, warm, edit, project-config edit). Default budgets scale by the larger of CI and local paired medians: fresh 9.9/15.4/28.8 s; cache cold 10.5/17.1/33.6 s; new project-config edit budget 11.2/18.0/37.0 s. The fresh ratio vs main keeps a temporary 1.25 wall, 1.41 CPU cold limit.

Verification

  • CI same-runner ratio (3 alternating pairs, 2 runs): default Rules cold 1.19 to 1.20 wall.
  • Local alternating pairs after a discarded warm-up: fresh 1.16 (3000, 5000), 1.12 (10000); cache 3000 cold 1.11, project-config 1.17. Raw samples and the exclusion rule are in COMPILE_BENCHMARKS.md.
  • Perf runner time about 125 to 104 min per PR.
  • fensu check reports 0 faults; ratio script unit tests pass.

…ess)

Checkpoint of the SQLClean integration: SQLCLEAN Rule codes, default
rule selection with the human-only notice, shared expansion inputs for
early lint, and SQL Rules skipped under --no-sql-analysis. e2e fixture
classification and Fensu faults are still open.
Pin sqlclean to v0.4.1, lint with the compile target and vars, skip
audit and SQL test files that opt out of SQL analysis, skip the row-rule
fact build for SQL-only selections, align fixtures and DuckShop with the
new findings, and update the docs.
@kvlonge
kvlonge enabled auto-merge (squash) November 1, 2026 10:25
@kvlonge
kvlonge disabled auto-merge November 1, 2026 10:26
…ligibility

SQL Rules now expand with the compiled project effective target and vars
on every compile path instead of falling back to the default target when
the early-lint reuse is missing. Commands without target flags use the
project default explicitly. Restore the eight-code format --fix allowlist
and its documented decisions, and cover --target and --vars end to end.
@kvlonge
kvlonge enabled auto-merge (squash) November 1, 2026 10:46
@kvlonge
kvlonge disabled auto-merge November 1, 2026 11:09
Default SQL Rules lint every compiled model, a measured 1.14x to 1.19x
cold wall cost over select = []. Compiler-only cases keep the previous
budgets and the 1.10 same-runner ratio; default-Rules cases scale the
cold budgets by the measured cost. Also pin sqlclean to the PIVOT USING
fix and keep two fixtures' derived tables out of SQLCLEAN039.
Pass compiler_only to the base generator when the base supports it, so
the compiler-only job keeps both sides without Rules once main runs them.
Mark the default-Rules cold allowance as temporary, reject
--compiler-only with the dense project at parse time, assert that
default-Rules guard cases run Rules, and pin sqlclean to the v0.4.2
release commit.
@kvlonge
kvlonge enabled auto-merge (squash) November 1, 2026 14:10
…ules

Default SQL Rules cost 1.09x on the 10,000-model compile and 1.20x and
1.25x on the 5,000- and 10,000-model cache guards' cold runs, measured
against select = [] on the same machine. Scale those cold budgets and
keep the previous budgets as compiler-only cases, as for 3,000 models.
@kvlonge
kvlonge disabled auto-merge November 1, 2026 14:53
…ts on CI

Replace the compiler-only twin at every guard size with one paired guard: the
3,000-model cache guard runs default Rules and select = [] in the same job and
covers cold, warm, edit and project-config edit compiles at the previous
budgets. Drop the compiler-only same-runner ratio job and the --compiler-only
option of the ratio script.

Scale each default Rules budget by the larger of the CI same-job and local
alternating medians, excluding pairs whose Rules-independent phases exceed
twice their median. Give the project-config edit its own budget, since it
reruns every rule.
… budget scope

The stated method gives 1.24x for the 5,000-model project-config edit, so its
budget is 18.0 s. The cold budget bounds the cold run as well as the
cache-disabled timeout.
@kvlonge
kvlonge merged commit f7c1e76 into main Nov 1, 2026
48 of 49 checks passed
@kvlonge
kvlonge deleted the feat/sqlclean-engine branch November 1, 2026 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant