Repository navigation
feat: exact KNN mesh magnification areas + fix partial-block neighbour search (arc phase 4b) #665
Description
Overview
Source & Cluster arc phase 4b. Phase 4a (#664) made every rectangular and Delaunay-family mesh's areas_for_magnification exact. KNearestNeighbor and KNNBarycentric still borrow the Delaunay dual areas, which aren't the integrals of their interpolants. Their identity-lens magnification is off by 1.6–4.4% (unit test) and up to 6% lensed. While measuring this I found #609, a neighbour-search bug in the last partial block, which corrupts KNN interpolation whenever the node count isn't a multiple of 128. This task fixes both.
Plan
- Fix fix: Preserve KNN neighbors in a partial final point block #609: pad the KNN node array to whole blocks so the last block holds the right points with the right indices. Add a brute-force oracle regression test at N=130.
- KNN and KNNBarycentric
areas_for_magnification= the integral of each node's interpolation weight over a uniform 256² grid masked to the convex hull (same domain as the Delaunay dual areas; host-side, like them). - Tighten the KNN tolerances in the PyAutoArray unit test and the autolens_workspace_test
imaging/mesh_magnification.pyscript to the new measurements. - DelaunayNN is left as is (within 0.5%) and recorded as the remaining approximation.
- Tier: judge — merge mode: human /prm.
Detailed implementation plan
Affected Repositories
- PyAutoArray (primary)
- autolens_workspace_test
Branch Survey
| Repository | Current Branch | Dirty? |
|---|---|---|
| ./PyAutoArray | main | clean |
| ./autolens_workspace_test | main | clean |
Suggested branch: feature/mesh-magnification-knn-areas
Implementation Steps
autoarray/inversion/mesh/interpolator/knn.py::get_interpolation_weights:jnp.padthe points ton_blocks * Brows and slice the padded array. The clampeddynamic_slicecurrently loads[N-B, N)while labelling the rowsstart + arange(B), so tail nodes are never seen and earlier nodes come back with wrong indices.- Add the KNN quadrature areas: an
areas_for_magnificationoverride for the KNN interpolators' mesh geometry. Evaluate the interpolator's own_mappings_sizes_weightson cell centres of a 256² grid over the vertex bounding box, keep only centres inside the convex hull (scipy.spatial.Delaunay.find_simplex), andbincountthe weights × cell area per node. - Tests: a fix: Preserve KNN neighbors in a partial final point block #609 oracle test (N=130, point_block=128, eager and jit, both KNN variants); tighter tolerances in
test_areas_for_magnification.py; area sum = hull area. - autolens_workspace_test
scripts/imaging/mesh_magnification.py: tighten the KNN tolerances to the new measurements.
Prototype (with #609 patched, 24×24 identity grid, 144 nodes)
| mesh | Delaunay dual areas | exact quadrature (n=200/400/800) |
|---|---|---|
| KNearestNeighbor | −3.99e-2 | −7e-6 / +1.6e-5 / +1.5e-5 |
| KNNBarycentric | −1.38e-2 | +1.4e-3 / +1.8e-3 / +1.6e-3 |
Key Files
autoarray/inversion/mesh/interpolator/knn.pyautoarray/inversion/mesh/mesh_geometry/delaunay.pytest_autoarray/inversion/pixelization/mesh_geometry/test_areas_for_magnification.pyautolens_workspace_test/scripts/imaging/mesh_magnification.py
Closes #609 on merge as well.
Original Prompt
Click to expand starting prompt
Mesh magnification: exact-quadrature areas for KNN meshes (phase 4b)
Type: feature
Target: autoarray
Repos:
- PyAutoArray
- autolens_workspace_test
Themes: - pixelization
- cluster
Difficulty: medium
Autonomy: supervised
Priority: high
Status: formalised
Consequence: judge
Review-minutes: 25
Epic: cluster-strong-lensing
Phase: 4b
Parent: draft/test/workspaces/mesh_magnification_correctness.md
Arc: draft/feature/autolens/source_cluster_arc.md
Filed: 2026-10-28
Human rulings (2026-10-28, verbatim answers to the 4a flagged decisions)
"1 I authroi,e 2 perform proper fix 3 add to 4b fix 4 file this yes"
- 2 = adaptive rectangular areas: perform the proper (exact hat-quadrature) fix.
- 3 = KNearestNeighbor / KNNBarycentric: add to this 4b fix.
- 4 = zeroed-ring guard-row mismatch filed separately as PyAutoArray#660 (not this phase).
NOTE: this renumbers the old parent's "4b" (library flux-ratio magnification promotion) — that and the
source_science recovery sweep become 4c/4d when issued.
Scope change (2026-10-29)
Human 2026-10-29 chose "Fix adaptive rect first": the adaptive rectangular exact hat quadrature
(work item 1 below) was folded INTO 4a (PyAutoArray#659, adaptive_rectangular_hat_areas_from)
before its PRs opened. Measured after: lensed BilinearAdaptImage +14.43% -> -0.57%, RTUAdaptImage
+8.56% -> +0.08%; identity BilinearAdaptDensity +11.23% -> -0.34%; tolerances tightened to 1%
(RTUAdaptDensity lensed stays -1.82%, unchanged by the areas). Open item from it: on a strongly
distorted toy (radial a=0.3, BilinearAdaptImage) the bbox-domain hat areas sit +6.5% / +13% vs the
traced-region quadrature (domain mismatch, separable interpolant extends into untraced bbox corners);
the lensed simulator check is unaffected. 4b is now items 2-4 (KNN / KNNBarycentric) only.
Measured bias at 4a (PyAutoArray#659; tolerances in 4a document these)
| mesh | unit μ_id−1 | integ. lensed | integ. identity |
|---|---|---|---|
| RectangularBilinearAdaptDensity | +3.1e-2 | −0.21% | +11.23% |
| RectangularBilinearAdaptImage | +2.4e-1 | +14.43% | +1.31% |
| RectangularRTUAdaptImage | +8.0e-2 | +8.56% | +1.05% |
| RectangularRTUAdaptDensity | +2.5e-3 | −1.82% | −0.24% |
| KNearestNeighbor | +1.6e-2 | +6.10% | −5.11% |
| KNNBarycentric | −4.4e-2 | −1.71% | −3.57% |
Work
- DONE IN 4a (see scope change). Adaptive rectangular (mesh_geometry/rectangular.py): replace node-midpoint CDF-transformed cell areas
with the exact per-axis quadrature of each node's bilinear hat through the CDF,
q(i) = ∫ hat_i(U(x)) dx, outer-product per axis. A 4a scratch prototype
(/tmp/claude-1000/-home-jammy-Code-PyAutoLabs/2aa32082-3125-4533-a9ef-6b0e392688b6/scratchpad/measure.py,
copy it into the task before /tmp is swept) took unit errors to ~1e-3 (Bilinear density
2.8e-2→2e-6, RTU image −6.5e-2→2e-3); one strongly distorted toy stayed +6.5% (suspected domain
mismatch: data region smaller than its bbox — confirm). Must stay JAX-jit/grad safe; check plotting
(edges_transformedconsumers) keeps drawing cell edges, i.e. split "areas" from "edges" if needed;
measure cost. - KNN / KNNBarycentric: areas = exact quadrature of their interpolants (∫ of the per-node Wendland /
barycentric-KNN weights), e.g. via the mapping-matrix column sums over a fine quadrature grid, or a
loud documented approximation if exact is infeasible — measure. - Tighten the 4a tolerances (unit test test_areas_for_magnification.py and autolens_workspace_test
scripts/imaging/mesh_magnification.py) to the new measurements (~1.5× measured, floor 1%). - RectangularUniform could optionally adopt the exact per-axis 9/8, 7/8 edge weights (review note);
negligible for interior sources.
Activity
Jammy2211 commentedon Oct 29, 2026
Scope change 2026-10-29 (human ruling): #609 is split out of this task. Its fix breaks the KNN + AdaptSplit gradient check (
jax_grad/knn.py); details and the patch are in #609. The exact KNN areas don't depend on it. They integrate whatever neighbours the interpolator selects, and measure KNN −0.23% / −0.12% and KNNBarycentric −0.40% / −0.14% (lensed / identity) against simulator truth either way. This task now ships the areas only.Jammy2211 commentedon Oct 29, 2026
Shipped and merged 2026-10-29:
e71feafd): KNearestNeighbor and KNNBarycentricareas_for_magnificationare now the integral of each vertex's basis function over the hull (MeshGeometryKNearestNeighbor).5a41b981): KNN tolerances inimaging/mesh_magnification.pytightened to 1%.Against simulator truth: KNN −0.23% / −0.12%, KNNBarycentric −0.40% / −0.14% (lensed / identity). #609 stays open separately. PyAutoArray change pending release.