Skip to content

fix: mask invalid mapper triplets on the imaging sparse path; lazy JAX import in its scatter-add helpers #643

Closed

Description

@Jammy2211

Overview

Side finding from #639 / PR #641. Every imaging sparse-inversion consumer (data vector, curvature diagonal and off-diagonal, mixed blocks, reconstructed image) reads Mapper.sparse_triplets_data / sparse_triplets_curvature, which are built by mapper_util.sparse_triplets_from without the slot-size and negative-index zeroing that the interferometer path applies in _sparse_triplets_slim_from. A -1 source index would silently wrap to the last source pixel and unused stencil slots contribute if their weights are non-zero. The two imaging scatter-add helpers also import JAX inside the function on every call, so the numpy branch receives JAX arrays and cannot run without JAX installed.

Plan

  • Centralise the masking in mapper_util.sparse_triplets_from behind an optional pix_sizes_for_sub argument; Mapper.sparse_triplets_data / sparse_triplets_curvature pass it; the interferometer _sparse_triplets_slim_from switches to the shared masking and drops its local copy (existing tests prove no change).
  • Make data_vector_via_psf_weighted_data_from and mapped_reconstructed_operated_data_via_sparse_operator_from dispatch on the backend: np.bincount on numpy, segment_sum under JAX, JAX imported only on that branch.
  • Tests: mapper-util test with a hand-built index table (negative index + unused slot with non-zero weight) zeroed on numpy and JAX; imaging sparse parity (data vector, curvature, reconstructed image) vs the dense mapping inversion on such a mapper; nojax-leg check that the imaging sparse numpy path imports and runs.
  • First step: establish whether any current mesh (Rectangular, Delaunay, DelaunayNN, Voronoi) emits negative indices or non-zero unused-slot weights; if none does, the PR body states the defect is latent and the change is a guard.
  • Tier: glance — merge mode: auto-merge on green if the Witness passes, post-merge summary
Detailed implementation plan

Affected Repositories

  • PyAutoArray (primary)

Branch Survey

Repository Current Branch Dirty?
array/PyAutoArray main clean (feature/interferometer-primary-beam in flight #640; #642 queued)

Suggested branch: feature/imaging-sparse-triplets-masking

Implementation Steps

  1. autoarray/inversion/mappers/mapper_util.py sparse_triplets_from (~270): add pix_sizes_for_sub=None; when given, build slot = tile(arange(P), M_sub), valid = (slot < repeat(pix_sizes, P)) & (cols >= 0), vals = where(valid, vals, 0), cols = where(valid, cols, 0) on both numpy (~328) and JAX (~355) branches.
  2. autoarray/inversion/mappers/abstract.py sparse_triplets_data / sparse_triplets_curvature (~440-470): pass pix_sizes_for_sub=self.pix_sizes_for_sub_slim_index.
  3. autoarray/inversion/inversion/interferometer/sparse.py _sparse_triplets_slim_from (~131-172): pass pix_sizes_for_sub, remove the local masking block; test_interferometer.py parity tests stay green.
  4. autoarray/inversion/inversion/imaging/inversion_imaging_util.py data_vector_via_psf_weighted_data_from (~70-104) and mapped_reconstructed_operated_data_via_sparse_operator_from (~280-303): add xp=np; numpy branch uses np.bincount(..., minlength=), JAX branch keeps segment_sum; callers in imaging/sparse.py (~90, ~144, ~168, ~526) pass xp=self._xp.
  5. Tests in test_autoarray/inversion/mappers/ (mapper_util) and test_autoarray/inversion/inversion/imaging/test_imaging.py; emulate the nojax leg (import-hook block) locally before ship.

Key Files

  • autoarray/inversion/mappers/mapper_util.py — shared triplet builder
  • autoarray/inversion/mappers/abstract.py — Mapper triplet properties
  • autoarray/inversion/inversion/imaging/inversion_imaging_util.py — scatter-add helpers
  • autoarray/inversion/inversion/imaging/sparse.py — consumers
  • autoarray/inversion/inversion/interferometer/sparse.py — local masking to retire

Original Prompt

Click to expand starting prompt

Imaging sparse inversion: unmasked mapper triplets in the reconstructed-image path and an unconditional JAX import

Type: bug
Target: PyAutoArray
Repos:

  • PyAutoArray
    Difficulty: small
    Autonomy: safe
    Priority: medium
    Status: formalised
    Consequence: glance
    Witness: a test asserting the imaging sparse mapped_reconstructed_data_dict equals mapping_matrix @ reconstruction for a mapper whose pix_indexes_for_sub_slim_index contains unused stencil slots (slot >= pix_sizes) and a negative index, on numpy without JAX importable (nojax leg) and under JAX
    Review-minutes: 3
    Unattended: ready
    Filed: 2026-10-21

Why

Side finding from PyAutoArray#639 (merged as #641, 2026-10-21). The interferometer sparse path zeroes invalid triplet slots in _sparse_triplets_slim_from (slot >= pix_sizes_for_sub_slim_index, or cols < 0 → weight 0, column 0). The imaging sparse path does not.

Where

  • autoarray/inversion/inversion/imaging/sparse.py around line 526, _mapped_reconstructed_data_dict_from, uses linear_obj.sparse_triplets_curvature, built in inversion/mappers/abstract.py:460-470 via mapper_util.sparse_triplets_from(..., return_rows_slim=False) with no masking. A -1 source index silently wraps to the last source pixel and unused slots contribute if their weights are non-zero.
  • The same helper, inversion_imaging_util.mapped_reconstructed_operated_data_via_sparse_operator_from (~line 280), imports JAX unconditionally even on the numpy path (nojax leg risk).
  • The imaging data-vector and curvature paths that consume the same unmasked Mapper.sparse_triplets_* likely share the gap; audit them in the same PR.

Fix

Apply the slot/negative-index masking once at the Mapper.sparse_triplets_* builders (or in sparse_triplets_from behind a flag) so every consumer is covered; make the JAX import lazy (xp-dispatched) in the imaging util. First establish whether any current mesh (Rectangular, Delaunay, DelaunayNN, Voronoi) actually emits negative indices or non-zero weights in unused slots; if none does, the fix is a guard plus the regression test, and the prompt notes the latent nature.

Activity

  1. Jammy2211 commented on Oct 21, 2026

    @Jammy2211
    CollaboratorAuthor

    Library PR created: #649 (head dd26822). Defect confirmed latent (Delaunay/DelaunayNN unused slots hold -1 with weight 0); masking is a prerequisite for the new numpy bincount branch. Follow-up: ImagingSparseOperator and the imaging curvature methods still import JAX unconditionally. Heart YELLOW acknowledged by the human 2026-10-21. Tier glance: auto-merge on green if the Witness passes.

  2. Jammy2211 commented on Oct 21, 2026

    @Jammy2211
    CollaboratorAuthor

    Shipped: #649 merged (glance tier, auto-merged on green: unittest 3.12 / 3.13 / nojax all pass; Witness masking + parity tests in-run). Pending release. Follow-up (not fixed here): ImagingSparseOperator.from_noise_map_and_psf and the imaging curvature methods import JAX unconditionally, so InversionImagingSparse still needs JAX on the numpy backend.

  3. Jammy2211 commented on Oct 21, 2026

    @Jammy2211
    CollaboratorAuthor

    Follow-up resolved without a change: ImagingSparseOperator is the JAX implementation by design (batched FFT via lax.fori_loop / dynamic_update_slice), with the NumPy/CPU alternative being the separately named Imaging.apply_sparse_operator_cpu → SparseLinAlgImagingNumba / InversionImagingSparseNumba (see the apply_sparse_operator docstring). The numpy branches added in #649 cover the only numpy-reachable helpers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions