Repository navigation
feat: primary beam for interferometer datasets, transformers and the sparse inversion #640
Description
Overview
Community proposal from @HRSAstro (PyAutoLabs/.github#35, accepted 2026-10-21). autoarray has no primary-beam handling, so interferometer inversions reconstruct the apparent sky P × true sky, which breaks surface-brightness conservation in lens modelling. With P on the image grid the response is A' = T P M, giving F' = Mᵀ P W̃ P M and D' = Mᵀ P D̃ while the w-tilde kernel, dirty image and dᵀN⁻¹d are unchanged. P lives on the Interferometer dataset and is applied in the transformers and the sparse operator so profiles, linear profiles and mappers all see the same P.
Plan
- P as an optional image on the
Interferometerdataset, validated against the real-space mask, with a Gaussian helper (FWHM 1.13 λ/D). - Applied inside both transformers on the slim grid before the scatter; the dirty-image adjoint stays un-beamed so kernels, dirty image and dirty beam are unchanged.
- Sparse w-tilde path: P folded into the mapper triplet weights (A' = P·A); linear-function blocks and profile terms get the same multiply; FFT kernels and SparseTerms untouched.
- Simulator accepts P; PyAutoGalaxy/PyAutoLens need no fitting change; persistence through analysis save/aggregator is a follow-up.
- Warn when any unmasked P is 0 (rank-deficient curvature); no floor.
- Tier: glance — merge mode: auto-merge on green if the Witness passes, post-merge summary
Detailed implementation plan
Affected Repositories
- PyAutoArray (primary)
- PyAutoGalaxy (verify only; persistence follow-up)
Branch Survey
| Repository | Current Branch | Dirty? |
|---|---|---|
| array/PyAutoArray | main | clean (feature/sparse-mapped-reconstructed-image in flight, #639) |
| galaxy/PyAutoGalaxy | main | clean |
Suggested branch: feature/interferometer-primary-beam
Implementation Steps
See the "What to build" and "Tests" sections of the prompt below (steps 1-11 with file paths and line anchors from the 2026-10-21 survey of main).
Key Files
autoarray/dataset/interferometer/primary_beam.py— new Gaussian helperautoarray/operators/transformer.py—primary_beam,with_primary_beam, slim multiply invisibilities_from/transform_mapping_matrixautoarray/dataset/interferometer/dataset.py— constructor arg, validation, warning, forwarding at every re-construction siteautoarray/inversion/inversion/interferometer/inversion_interferometer_util.py— operator field,sparse_profile_terms_fromautoarray/inversion/pixelization/mappers/mapper_util.py—slim_weightsonsparse_triplets_fromautoarray/inversion/inversion/interferometer/sparse.py— pass P into triplets and linear-func blocksautoarray/fit/fit_interferometer.py—dirty_model_image_natural_fromautoarray/dataset/interferometer/simulator.py—primary_beamarg
Original Prompt
Click to expand starting prompt
Primary beam correction for interferometer datasets, transformers and the sparse w-tilde inversion
Type: feature
Target: PyAutoArray
Repos:
- PyAutoArray
- PyAutoGalaxy
Difficulty: large
Autonomy: supervised
Priority: high
Community: Include a primary beam correction for interferometer inversions .github#35
Status: formalised
Consequence: glance
Witness: tests asserting (i) primary_beam=None and primary_beam=ones leave transformer outputs, sparse and mapping curvature, data vector and fast_chi_squared bit-identical on numpy and under JAX jit, and (ii) with a non-trivial P the sparse curvature equals the brute-force Mᵀ diag(P) W̃ diag(P) M and the data vector equals Mᵀ diag(P) D̃ to 1e-10 on the 7×7 Delaunay fixture
Review-minutes: 8
Unattended: ready
Filed: 2026-10-21
user-facing: true
Why
Community proposal by @HRSAstro (Ideas & Proposals, 2026-10-07, accepted 2026-10-21). autoarray has no primary-beam handling: the transformer maps the image straight to visibilities, so an inversion reconstructs the apparent sky (P × true sky), which breaks surface-brightness conservation for lens modelling. With P on the image grid the response becomes A' = T P M, so F' = Mᵀ P W̃ P M and D' = Mᵀ P D̃; the w-tilde kernel, dirty image and dᵀN⁻¹d are unchanged. The maintainer's reply fixed the design: P lives on the Interferometer dataset as an optional image on the real-space mask and is applied inside both the transformer and the sparse operator, so light profiles, linear light profiles and mappers all see the same P; user-supplied beam image is the core API with a Gaussian helper (FWHM ≈ 1.13 λ/D); mosaics deferred. The contributor's regularisation-wrapper form (P⁻¹ H P⁻¹ on the mesh) is only exact when P is constant across a mesh pixel and evaluates P in the wrong plane for lensed sources, so it is not adopted, but remains expressible via a custom regularisation. Reference snippet: https://github.com/user-attachments/files/33762670/upstream_snippets.py section 7 (primary_beam_map, pb_correct).
What to build (survey of main, 2026-10-21; paths under autoarray/)
- New
dataset/interferometer/primary_beam.py:primary_beam_gaussian_from(mask, frequency_hz, dish_diameter_m, pb_factor=1.13, centre=(0.0, 0.0)) -> Array2D(FWHM = pb_factor·c/ν/D in arcsec, peak 1 at centre, coordinates viaGrid2D.from_mask); export fromautoarray/__init__.py. operators/transformer.py:primary_beam=Noneon both transformers, stored slim;with_primary_beam(P)shallow copy; multiply on the slim grid before the scatter invisibilities_fromandtransform_mapping_matrix(DFT and NUFFT, numpy and JAX).image_fromstays the un-beamed adjoint (D̃ must not change); docstring says so.dataset/interferometer/dataset.py:primary_beam: Optional[Array2D] = Noneconstructor arg, shape-validated againstreal_space_mask(native or slim accepted),primary_beam_slimproperty; after the transformer is built setself.transformer = self.transformer.with_primary_beam(P)(idempotent, safe for lambda-returned transformers); forward P in the pooled-noise branch (~599),apply_sparse_operator(~699),apply_sparse_operator_from_chunks(~865),from_sparse_terms/from_stream/from_fits(primary_beam_path); passprimary_beam_sliminto the operator constructors (~691, ~859).inversion/inversion/interferometer/inversion_interferometer_util.py:InterferometerSparseOperatorgainsprimary_beam_slim: Optional[np.ndarray] = None, threaded throughfrom_nufft_precision_operatorandfrom_sparse_terms(SparseTerms and chunk accumulation are beam-independent, unchanged);sparse_profile_terms_fromappliesimage ← P·image(~2163) so sparse_dirty_image = D̃ − W̃ P i and data_term = dᵀN⁻¹d − 2(Pi)ᵀD̃ + (Pi)ᵀW̃(Pi).inversion/pixelization/mappers/mapper_util.py:slim_weights=Nonekwarg onsparse_triplets_from,vals *= slim_weights[slim_rows](numpy ~332, JAX ~358), so A' = P·A and the FFT kernels/preloads stay untouched.inversion/inversion/interferometer/sparse.py: readP = getattr(self.dataset.sparse_operator, "primary_beam_slim", None)(works for PyAutoGalaxy's DatasetInterface); pass asslim_weightsin_sparse_triplets_slim_fromand_sparse_triplets_curvature_from; scale linear-func mapping matrices byP[:, None]in the data vector (~123) and mixed blocks (~443-445). Model visibilities viatransformer.visibilities_frompick P up automatically;mapped_reconstructed_data_dictstays the un-beamed true-sky image.fit/fit_interferometer.py: apply P indirty_model_image_natural_from(~52).dataset/interferometer/simulator.py:primary_beam=Nonearg, add to the pytreeno_flattenlist,with_primary_beamon the built transformer, forward P to the returned Interferometer.- P = 0 pixels: emit a
warnings.warnfrom the dataset when any unmasked P value is 0 (curvature rank-deficient for mesh pixels mapping only there; regularisation absorbs it); no floor is applied. - Dense path:
operated_mapping_matrix_overridedocstring states P is the caller's job. - PyAutoGalaxy: no fitting changes (DatasetInterface receives the dataset's transformer and sparse_operator). Follow-up prompt: persist P in
analysis.pysave_attributes and the aggregator rebuild.
Tests
- P=None / P=ones parity, bit-for-bit (
np.array_equal) on numpy and jit, extending test_interferometer.py ~878 and ~1349. - Brute force on the 7×7 Delaunay fixture with a non-trivial P: dense W̃ from DFT
transform_mapping_matrix(identity)with P=None; sparse curvature vs Mᵀ diag(P) W̃ diag(P) M; data vector vs Mᵀ diag(P) D̃; func_list+mapper, x2 mappers, func-list-only. - Dense (mapping) vs sparse with P, DFT vs NUFFT, array-free
from_sparse_terms(primary_beam=)vs in-memory. - Transformer:
visibilities_from(img)with P equals un-beamedvisibilities_from(P·img); same for mapping matrix; DFT, NUFFT, JAX. - Dataset propagation through all four re-construction sites; shape mismatch raises;
dirty_image/dirty_beamunchanged by P. - Simulator: P-simulated visibilities equal un-beamed simulation of P·image. Gaussian helper: peak 1 at centre, 0.5 at FWHM/2.
Acceptance
- Witness passes; nojax leg imports clean. Workspace example (
interferometer/advancedprimary-beam snippet) and docs as a follow-up workspace prompt after the library PR ships. Update the contributor on the thread at PR-open and at release.
Activity
Jammy2211 commentedon Oct 21, 2026
Library PR created: #644 (branch feature/interferometer-primary-beam, head 8c73a9f). Workspace impact: additive API only (option i: a workspace example + docs follow as their own prompt). Heart YELLOW acknowledged by the human 2026-10-21 (manifest drift 1 mismatch; release validation stale after #641). Tier glance: auto-merge on green if the Witness passes.
Jammy2211 commentedon Oct 21, 2026
Shipped: #644 merged (glance tier, auto-merged on green: unittest 3.12 / 3.13 / nojax all pass; Witness bit-identity + brute-force parity tests in-run). Primary beam on
Interferometer, transformers, sparse operator and simulator; Gaussian helperaa.primary_beam_gaussian_from. Pending release. Follow-ups filed in Mind: workspace example + docs; PyAutoGalaxy persistence (analysis save + aggregator).