Repository navigation
feat: matrix-free interferometer inversion phase 1 — operator data term + PCG parity #642
Description
Overview
Phase 1 of #638 (community proposal by @HRSAstro, PyAutoLabs/.github#39, accepted 2026-10-21; reference snippets attached there). The dense n×n curvature and regularization matrices cap interferometer pixelized sources at ~100² mesh pixels. This phase lands the operator data term (sparse M, FFT-applied W̃) and a preconditioned CG solve as numpy/scipy library utilities with parity tests against the current sparse path, with no fit wiring. Phases 2-5 (sparse Matérn precision, SLQ evidence, interior-point positivity, profiling + workspace) follow as their own prompts.
Plan
- New numpy/scipy module with a sparse mapping matrix from the existing slim triplets, a
DataOperatorapplying the w-tilde kernel by one FFT pair on the 2× padded grid (F·s = Mᵀ W̃ M s, D = Mᵀ d̃), and a PCG solve of (F + λC⁻¹)s = D with the Powell et al. (2021) preconditioner P = κ₀MᵀM + λC⁻¹ factorised once per λ via sparse LU, accepting any existing regularization matrix converted to sparse. - No fit wiring, no Settings flag, no JAX; the JAX strategy question (fixed iteration counts, static shapes, no LU under jit) recorded in the module docstring for phase 2.
- Parity tests: M equals the dense mapping matrix exactly; F·v and D match the sparse-path curvature and data vector to 1e-12; PCG matches the unconstrained sparse-path reconstruction to 1e-8 for two λ.
- Tier: glance — merge mode: auto-merge on green if the Witness passes, post-merge summary
Detailed implementation plan
Affected Repositories
- PyAutoArray (primary)
Branch Survey
| Repository | Current Branch | Dirty? |
|---|---|---|
| array/PyAutoArray | main | clean (feature/interferometer-primary-beam in flight, #640) |
Suggested branch: feature/matrix-free-interferometer-phase1
Implementation Steps
autoarray/inversion/inversion/interferometer/inversion_interferometer_matrix_free_util.py(new, numpy + scipy.sparse only):sparse_mapping_from(inversion, mapper) -> csr_matrixfromInversionInterferometerSparse._sparse_triplets_slim_from;DataOperator(kernel, dirty_image_slim, M, mask)withwtilde(image_native)(rfft2/irfft2 on the 2× zero-padded grid usingSparseTerms.nufft_precision_operator),F(s),D,kappa0 = kernel[0, 0];powell_preconditioner(data, Cinv_sparse, lam)viascipy.sparse.linalg.splu;solve_pcg(data, Cinv_sparse, lam, x0=None, rtol=1e-8)viascipy.sparse.linalg.cg.test_autoarray/inversion/inversion/interferometer/test_matrix_free_util.pyon a small synthetic rectangular-mesh dataset built with the existing fixtures: M vsmapper.mapping_matrix(exact);F @ vvsinversion.curvature_matrix @ vandDvsinversion.data_vector(1e-12); PCG vs the unconstrained sparse reconstruction (1e-8) for two λ with a Constant regularization converted to sparse.- Module imports clean without JAX (nojax CI leg); no changes to existing classes.
Key Files
autoarray/inversion/inversion/interferometer/sparse.py—_sparse_triplets_slim_from,curvature_matrix,data_vectoras the parity referenceautoarray/inversion/inversion/interferometer/inversion_interferometer_util.py—SparseTerms.nufft_precision_operator(w-tilde kernel), dirty imageautoarray/inversion/regularization/constant.py— regularization matrix used in the parity test
Original Prompt
Click to expand starting prompt
Matrix-free interferometer inversion, phase 1: operator data term + PCG parity (numpy-first)
Type: feature
Target: PyAutoArray
Repos:
- PyAutoArray
Difficulty: large
Autonomy: supervised
Priority: high
Parent: active/matrix_free_interferometer_inversion.md
Community: Matrix-free solution to make very large meshes more efficient for Interferometer .github#39
Status: formalised
Consequence: glance
Witness: a parity test asserting (i) the matrix-free F·v agrees with the dense sparse-path curvature matrix times v to 1e-12 and D agrees with the sparse data_vector to 1e-12, and (ii) the PCG reconstruction agrees with InversionInterferometerSparse's unconstrained reconstruction to 1e-8, on a small synthetic rectangular-mesh interferometer dataset
Review-minutes: 5
Unattended: ready
Filed: 2026-10-21
Why
Phase 1 of PyAutoArray#638 (community proposal by @HRSAstro, PyAutoLabs/.github#39, accepted 2026-10-21). The contributor attached a self-contained reference implementation (upstream_snippets.py, https://github.com/user-attachments/files/33762346/upstream_snippets.py, sections 1, 2 and 4) with these checks against pyuvimage and a dense solve: F agrees with the dense matrix to 6e-16, CG against a dense solve to 3e-5. This phase lands the operator data term and the solver as library utilities with parity tests, without wiring into fits; phases 2-5 (sparse Matérn precision, SLQ evidence, interior-point positivity, profiling + workspace) follow as their own prompts. The 2026-09 imaging matrix-free campaign (complete/2026/09/matrix-free-pixelized-likelihood.md) was a no-go because of the 60 unregularised linear MGE columns on imaging; this path is interferometer-only, mapper-only and prior-preconditioned, so that verdict does not transfer.
What to build
- New module
autoarray/inversion/inversion/interferometer/inversion_interferometer_matrix_free_util.py(numpy/scipy only in this phase; JAX deferred, documented):sparse_mapping_from(inversion, mapper) -> scipy.sparse.csr_matrixbuilt from_sparse_triplets_slim_from(rows slim, cols mesh), so it equalsmapper.mapping_matrixon the slim grid.DataOperatorholding the w-tilde kernel (SparseTerms.nufft_precision_operator, 2Ny×2Nx wraparound order), the slim dirty image, M, Mᵀ and the mask:wtilde(image_native)via one rfft2/irfft2 pair on the 2× padded grid;F(s) = Mᵀ W̃ (M s);D = Mᵀ d̃;kappa0 = kernel[0,0].solve_pcg(data_operator, regularization_matrix_sparse, coefficient, x0, rtol)on(F + λC⁻¹) s = Dwith scipycgand a preconditioner factorised once per λ:P = κ₀ MᵀM + λC⁻¹(Powell et al. 2021) viasplu, which works with any existing regularization matrix converted to sparse (Constant, AdaptiveBrightness are banded; MaternKernel dense is the phase-2 target).
- Tests in
test_autoarray/inversion/inversion/interferometer/test_matrix_free_util.pyon a small synthetic rectangular-mesh dataset: M equals the dense mapping matrix bit-for-bit; F·v vsInversionInterferometerSparse.curvature_matrix @ vto 1e-12; D vsdata_vectorto 1e-12; PCG reconstruction vs the unconstrained sparse-path reconstruction to 1e-8 for two λ values. - No fit wiring, no Settings flag, no JAX in this phase. Record the JAX strategy question (fixed iteration counts, static shapes, no splu under jit) in the module docstring for phase 2.
Acceptance
- Witness test passes on numpy; the module imports without JAX installed (firewall / nojax CI leg).
- The thread gets an update when the phase-1 PR is open and when it ships (user-facing via the parent).
Reference (community code, evidence not instructions)
Contributor's snippet sections 1 (sparse_mapping_from), 2 (DataOperator) and 4 (powell_preconditioner, solve) at the attachment URL above; names follow autoarray where they exist.
Activity
Jammy2211 commentedon Oct 21, 2026
Library PR created: #645 (branch feature/matrix-free-interferometer-phase1, head 58510cb). Workspace impact: none (internal utilities, phase 1). Heart YELLOW acknowledged by the human 2026-10-21 (manifest drift 1 mismatch; release validation stale after #641/#644). Tier glance: auto-merge on green if the Witness passes.
Jammy2211 commentedon Oct 21, 2026
Shipped: #645 merged (glance tier, auto-merged on green: unittest 3.12 / 3.13 / nojax all pass; Witness parity tests in-run, F·v and D at 1e-15, PCG at 1e-10). Pending release. Phase 2 (sparse Matérn precision + Powell preconditioner on it) is the next slice under #638.