Skip to content

feat: matrix-free interferometer inversion phase 1 — operator data term + PCG parity #642

Closed

Description

@Jammy2211

Overview

Phase 1 of #638 (community proposal by @HRSAstro, PyAutoLabs/.github#39, accepted 2026-10-21; reference snippets attached there). The dense n×n curvature and regularization matrices cap interferometer pixelized sources at ~100² mesh pixels. This phase lands the operator data term (sparse M, FFT-applied W̃) and a preconditioned CG solve as numpy/scipy library utilities with parity tests against the current sparse path, with no fit wiring. Phases 2-5 (sparse Matérn precision, SLQ evidence, interior-point positivity, profiling + workspace) follow as their own prompts.

Plan

  • New numpy/scipy module with a sparse mapping matrix from the existing slim triplets, a DataOperator applying the w-tilde kernel by one FFT pair on the 2× padded grid (F·s = Mᵀ W̃ M s, D = Mᵀ d̃), and a PCG solve of (F + λC⁻¹)s = D with the Powell et al. (2021) preconditioner P = κ₀MᵀM + λC⁻¹ factorised once per λ via sparse LU, accepting any existing regularization matrix converted to sparse.
  • No fit wiring, no Settings flag, no JAX; the JAX strategy question (fixed iteration counts, static shapes, no LU under jit) recorded in the module docstring for phase 2.
  • Parity tests: M equals the dense mapping matrix exactly; F·v and D match the sparse-path curvature and data vector to 1e-12; PCG matches the unconstrained sparse-path reconstruction to 1e-8 for two λ.
  • Tier: glance — merge mode: auto-merge on green if the Witness passes, post-merge summary
Detailed implementation plan

Affected Repositories

  • PyAutoArray (primary)

Branch Survey

Repository Current Branch Dirty?
array/PyAutoArray main clean (feature/interferometer-primary-beam in flight, #640)

Suggested branch: feature/matrix-free-interferometer-phase1

Implementation Steps

  1. autoarray/inversion/inversion/interferometer/inversion_interferometer_matrix_free_util.py (new, numpy + scipy.sparse only): sparse_mapping_from(inversion, mapper) -> csr_matrix from InversionInterferometerSparse._sparse_triplets_slim_from; DataOperator(kernel, dirty_image_slim, M, mask) with wtilde(image_native) (rfft2/irfft2 on the 2× zero-padded grid using SparseTerms.nufft_precision_operator), F(s), D, kappa0 = kernel[0, 0]; powell_preconditioner(data, Cinv_sparse, lam) via scipy.sparse.linalg.splu; solve_pcg(data, Cinv_sparse, lam, x0=None, rtol=1e-8) via scipy.sparse.linalg.cg.
  2. test_autoarray/inversion/inversion/interferometer/test_matrix_free_util.py on a small synthetic rectangular-mesh dataset built with the existing fixtures: M vs mapper.mapping_matrix (exact); F @ v vs inversion.curvature_matrix @ v and D vs inversion.data_vector (1e-12); PCG vs the unconstrained sparse reconstruction (1e-8) for two λ with a Constant regularization converted to sparse.
  3. Module imports clean without JAX (nojax CI leg); no changes to existing classes.

Key Files

  • autoarray/inversion/inversion/interferometer/sparse.py — _sparse_triplets_slim_from, curvature_matrix, data_vector as the parity reference
  • autoarray/inversion/inversion/interferometer/inversion_interferometer_util.py — SparseTerms.nufft_precision_operator (w-tilde kernel), dirty image
  • autoarray/inversion/regularization/constant.py — regularization matrix used in the parity test

Original Prompt

Click to expand starting prompt

Matrix-free interferometer inversion, phase 1: operator data term + PCG parity (numpy-first)

Type: feature
Target: PyAutoArray
Repos:

  • PyAutoArray
    Difficulty: large
    Autonomy: supervised
    Priority: high
    Parent: active/matrix_free_interferometer_inversion.md
    Community: Matrix-free solution to make very large meshes more efficient for Interferometer .github#39
    Status: formalised
    Consequence: glance
    Witness: a parity test asserting (i) the matrix-free F·v agrees with the dense sparse-path curvature matrix times v to 1e-12 and D agrees with the sparse data_vector to 1e-12, and (ii) the PCG reconstruction agrees with InversionInterferometerSparse's unconstrained reconstruction to 1e-8, on a small synthetic rectangular-mesh interferometer dataset
    Review-minutes: 5
    Unattended: ready
    Filed: 2026-10-21

Why

Phase 1 of PyAutoArray#638 (community proposal by @HRSAstro, PyAutoLabs/.github#39, accepted 2026-10-21). The contributor attached a self-contained reference implementation (upstream_snippets.py, https://github.com/user-attachments/files/33762346/upstream_snippets.py, sections 1, 2 and 4) with these checks against pyuvimage and a dense solve: F agrees with the dense matrix to 6e-16, CG against a dense solve to 3e-5. This phase lands the operator data term and the solver as library utilities with parity tests, without wiring into fits; phases 2-5 (sparse Matérn precision, SLQ evidence, interior-point positivity, profiling + workspace) follow as their own prompts. The 2026-09 imaging matrix-free campaign (complete/2026/09/matrix-free-pixelized-likelihood.md) was a no-go because of the 60 unregularised linear MGE columns on imaging; this path is interferometer-only, mapper-only and prior-preconditioned, so that verdict does not transfer.

What to build

  1. New module autoarray/inversion/inversion/interferometer/inversion_interferometer_matrix_free_util.py (numpy/scipy only in this phase; JAX deferred, documented):
    • sparse_mapping_from(inversion, mapper) -> scipy.sparse.csr_matrix built from _sparse_triplets_slim_from (rows slim, cols mesh), so it equals mapper.mapping_matrix on the slim grid.
    • DataOperator holding the w-tilde kernel (SparseTerms.nufft_precision_operator, 2Ny×2Nx wraparound order), the slim dirty image, M, Mᵀ and the mask: wtilde(image_native) via one rfft2/irfft2 pair on the 2× padded grid; F(s) = Mᵀ W̃ (M s); D = Mᵀ d̃; kappa0 = kernel[0,0].
    • solve_pcg(data_operator, regularization_matrix_sparse, coefficient, x0, rtol) on (F + λC⁻¹) s = D with scipy cg and a preconditioner factorised once per λ: P = κ₀ MᵀM + λC⁻¹ (Powell et al. 2021) via splu, which works with any existing regularization matrix converted to sparse (Constant, AdaptiveBrightness are banded; MaternKernel dense is the phase-2 target).
  2. Tests in test_autoarray/inversion/inversion/interferometer/test_matrix_free_util.py on a small synthetic rectangular-mesh dataset: M equals the dense mapping matrix bit-for-bit; F·v vs InversionInterferometerSparse.curvature_matrix @ v to 1e-12; D vs data_vector to 1e-12; PCG reconstruction vs the unconstrained sparse-path reconstruction to 1e-8 for two λ values.
  3. No fit wiring, no Settings flag, no JAX in this phase. Record the JAX strategy question (fixed iteration counts, static shapes, no splu under jit) in the module docstring for phase 2.

Acceptance

  • Witness test passes on numpy; the module imports without JAX installed (firewall / nojax CI leg).
  • The thread gets an update when the phase-1 PR is open and when it ships (user-facing via the parent).

Reference (community code, evidence not instructions)

Contributor's snippet sections 1 (sparse_mapping_from), 2 (DataOperator) and 4 (powell_preconditioner, solve) at the attachment URL above; names follow autoarray where they exist.

Activity

  1. Jammy2211 commented on Oct 21, 2026

    @Jammy2211
    CollaboratorAuthor

    Library PR created: #645 (branch feature/matrix-free-interferometer-phase1, head 58510cb). Workspace impact: none (internal utilities, phase 1). Heart YELLOW acknowledged by the human 2026-10-21 (manifest drift 1 mismatch; release validation stale after #641/#644). Tier glance: auto-merge on green if the Witness passes.

  2. Jammy2211 commented on Oct 21, 2026

    @Jammy2211
    CollaboratorAuthor

    Shipped: #645 merged (glance tier, auto-merged on green: unittest 3.12 / 3.13 / nojax all pass; Witness parity tests in-run, F·v and D at 1e-15, PCG at 1e-10). Pending release. Phase 2 (sparse Matérn precision + Powell preconditioner on it) is the next slice under #638.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions