Repository navigation
Description
Overview
With use_positive_only_solver=False on the NumPy backend, every likelihood call factorizes the same symmetric positive-definite matrix F + H twice: once with np.linalg.solve (LU) for the reconstruction and once with np.linalg.cholesky for log_det_curvature_reg_matrix_term. On the Aris SPT2319-55 interferometer setup (Delaunay + AdaptSplit, numba route) these two take 31% of a call at 1,405 vertices and 72% at 2,947 vertices. The positive-only path already reuses its factor. This task makes the positive-negative path factorize once and use that factor for both the solve and the log det.
Campaign: PyAutoLabs/PyAutoPulse#62 (task interferometer_large_mask_aris).
Plan
- Add one cached Cholesky factor of
F + Hto the inversion, built only on the NumPy backend, withuse_positive_only_solver=Falseandlog_det_method == "cholesky". reconstructionsolves with that factor (cho_solve) in place ofnp.linalg.solve.log_det_curvature_reg_matrix_termreads2 * sum(log(diag(factor)))off the same factor when every linear object is regularized, so the reduced matrix is the full one.- If the Cholesky fails (matrix not positive-definite, or non-finite), nothing changes. The reconstruction falls back to the existing LU solve with its test-mode guard, and the log det takes the existing dense route, which raises or returns 0.0 in test mode as today. Error behaviour is therefore identical.
- Leave untouched: the JAX path, the positive-only path, the
slogdetopt-in, and the out-of-placecurvature_reg_matrix(it is reused by the reduced matrix and the covariance, so it is not factorized in place). No float32. - Tests pin the solve and log det against the two-factorization values, the fallbacks,
slogdetand positive-only. A laptop witness measures before and after at 1,405 and 2,947 vertices.
Tier: glance — merge mode: auto-merge on green if the Witness passes, post-merge summary
Detailed implementation plan
Affected Repositories
- PyAutoArray (primary)
Branch Survey
| Repository | Current Branch | Dirty? |
|---|---|---|
| array/PyAutoArray | main (17caf22 = origin/main) | clean |
worktree_check_conflict reuse-cholesky-factor PyAutoArray → no conflict.
Suggested branch: feature/reuse-cholesky-factor
Implementation Steps
autoarray/inversion/inversion/abstract.py: add@cached_property _curvature_reg_matrix_cho_factor. It returnsscipy.linalg.cho_factor(curvature_reg_matrix, lower=True, check_finite=False)whenself._xp is np,not settings.use_positive_only_solverandsettings.log_det_method == "cholesky". It returnsNoneunder any other gate, onLinAlgError(scipy's is numpy's class), or when the factor's diagonal is non-finite. It never overwritescurvature_reg_matrix.reconstruction: in the positive-negative branch, if the factor is present, returnscipy.linalg.cho_solve(factor, data_vector, check_finite=False). Otherwise call the unchangedinversion_util.reconstruction_positive_negative_from.log_det_curvature_reg_matrix_term: after the positive-only fast path, ifall_linear_obj_have_regularizationand the factor is present, return2.0 * np.sum(np.log(np.diag(factor[0]))). Otherwise take the unchanged_log_det_symmetric_from(curvature_reg_matrix_reduced). The log det reads the factor property directly, so it still does not evaluate the reconstruction.- Tests (new
test_autoarray/inversion/inversion/test_positive_negative_shared_cholesky.py): imaging and interferometer fixture inversions, where the reconstruction equalsnp.linalg.solveto rtol 1e-10 and the log det equals the dense Cholesky to 1e-9, and the factor is computed once and shared; non-PD (indefinite, nonsingular)F + Hwhere the reconstruction equals the LU solution and the log det raisesLinAlgError(test mode off) / returns 0.0 (test mode on);slogdetunchanged and factorNone; positive-only unaffected and factorNone; a partially regularized system where the solve uses the factor and the log det takes the dense reduced route. - Witness: Aris
bd.pybreakdown at 1,405 (--mesh step2) and 2,947 (--mesh sp) vertices, single thread, before (canonical main) and after (worktree). Record the full-call median and the solve+log-det share, logL agreement (1e-9) and the reconstruction max relative difference. Pass: ≥15% at 1,405 and ≥35% at 2,947.
Key Files
autoarray/inversion/inversion/abstract.py:reconstruction,log_det_curvature_reg_matrix_term, new factor propertyautoarray/inversion/inversion/inversion_util.py:reconstruction_positive_negative_from(unchanged; fallback)
Heart RED development override
Heart was RED at launch (2026-10-15 late): Heart: RED — Release RED; monitoring RED · 10/100 · 415 unresolved; [red] release validation FAILED (stage integrate); [red] PyAutoHands/workflows/Python Version Matrix: failure; [red] PyAutoHeart/workflows/Release Integrate: failure; [red] PyAutoMind/workflows/Lifecycle Drift: failure; [red] PyAutoArray/Tests: red; [red] autofit_workspace_test/Smoke Tests: red; [red] autogalaxy_workspace_test/Smoke Tests: red; [red] autolens_workspace_test/Smoke Tests: red; [red] Release validation: NOT release_ready — v2026.10.15.1.dev81701; [red] Worktree drift: 5 orphan / 0 missing / 4 dirty; [red] autolens_inference: red; [red] autolens_profiling: red; [red] euclid_strong_lens_modeling_pipeline: red (as listed on PyAutoArray#626).
The live human authorized the development-only override in session 3e3dea38 (AskUserQuestion answer: "L1 now under override (Recommended)", 2026-10-15 late). Permitted: commit, push, PR-open behind the autonomous-ship gate (tests, smoke and review must still pass). Merge stays a separate human act. Heart remains RED for release purposes. Launched --auto.
Original Prompt
Click to expand starting prompt
Reuse one Cholesky factor of F+H for the reconstruction solve and the log det (numpy, non-positive-only path)
Type: feature
Target: PyAutoArray
Repos:
- PyAutoArray
Difficulty: small
Autonomy: safe
Priority: high
Status: formalised
Consequence: glance
Witness: on the Aris 125x125 interferometer setup (Delaunay + AdaptSplit, use_positive_only_solver=False, numba route) one likelihood call drops by >=15% at 1,405 vertices and >=35% at 2,947 vertices, with reconstruction unchanged to 1e-10 relative and log_det_curvature_reg_matrix_term unchanged to 1e-9
Review-minutes: 3
Unattended: ready
Profiled 2026-10-15 (session 3e3dea38, PyAutoArray main 17caf22) on Aris's SPT2319-55 data with
use_positive_only_solver=False: the numpy path factorizes the same SPD matrix F + H twice per
likelihood call — np.linalg.solve (LU) in inversion_util.reconstruction_positive_negative_from
(called from AbstractInversion.reconstruction, abstract.py ~760) and np.linalg.cholesky in
_log_det_symmetric_from for log_det_curvature_reg_matrix_term. Together 31% of the call at 1,405
vertices (100 + 70 ms of 0.63 s) and 72% at 2,947 vertices (976 + 741 ms of 2.0–2.4 s). The
positive-only path already reuses its factor (log_det_from_passive_cholesky_from); the non-positive
path does not.
Do: when use_positive_only_solver=False on the numpy backend, compute one scipy.linalg.cho_factor
of F + H, solve with cho_solve, and take the log det as 2 * sum(log(diag)) of the same factor
(keep the existing LinAlgError test-mode guard and the slogdet opt-in; JAX path unchanged).
Measured on the saved matrices: 65 ms vs 113+81 ms (1,405) and 612 ms vs 948+654 ms (2,947);
solution relative change 7e-14 / 2e-11; log det identical; non-PD still raises. Optionally fold in
an in-place F + H add to drop one S×S copy (10–30 ms). Do NOT use float32 (0.027 nats log-det
error at 2,947; cond(F+H) up to 7e12). Add a test pinning solve + log det against the two-factorization
values. Evidence: session 3e3dea38 scratchpad report_aris_speedups.md + aris_research/la.py
(temporary); campaign PyAutoLabs/PyAutoPulse#62.
Activity
Jammy2211 commentedon Oct 15, 2026
Workspace impact: (iii) none
API changes: none (internal only). The new
_curvature_reg_matrix_cho_factoris private, andreconstruction/log_det_curvature_reg_matrix_termkeep their signatures and semantics.no_run.yamlwas not checked: it is not present atPyAutoHands/autohands/config/in this checkout. The symbols are private and unused by scripts, so this is moot.Recommendation: no
/start_workspacefollow-up.Jammy2211 commentedon Oct 15, 2026
Library PR Created
pending-release, head 02aa2e4)Gate
Witness
Next: the human runs
/prm. No workspace follow-up is needed (option iii). The run stops at PR-open.Jammy2211 commentedon Oct 15, 2026
Shipped
Merged in #634 (e59d5fa). It is pending release and reaches users with the next PyAutoArray release.
log_det_methodcholesky, one cachedscipy.linalg.cho_factorof a copy of F+H now serves bothreconstruction(viacho_solve) andlog_det_curvature_reg_matrix_term. Every other configuration keeps the old path, and so does a factorization failure or a non-finite diagonal. Error semantics are unchanged. 10 new tests.curvature_reg_matrixandcurvature_matrixare reused elsewhere.interferometer_large_mask_aris).complete/2026/10/reuse-cholesky-factor.md.