JP-2928: Update regression test data for NIRCam subarray

[melanieclarke]

Partially addresses JP-2928

Update regression test data for NIRCam subarray, reference pixel correction test.

The new data is from PID 1409, obs 34, chosen for its similarity to the original data product (subarray subgrism64, exp_type nrc_tsimage, ngroups 5, nints 1).

The new data is from PID 2459, obs 5: EXP_TYPE=NRC_TSIMAGE, SUBARRAY=SUBGRISM64, NINTS=4, NGROUPS=5, NOUTPUTS=4.

Tasks

• request a review from someone specific, to avoid making the maintainers review every PR
• add a build milestone, i.e. Build 11.3 (use the latest build if not sure)
• Does this PR change user-facing code / API? (if not, label with no-changelog-entry-needed)
  • write news fragment(s) in changes/: echo "changed something" > changes/<PR#>.<changetype>.rst (see below for change types)
  • update or add relevant tests
  • update relevant docstrings and / or docs/ page
  • start a regression test and include a link to the running job (click here for instructions)
    • Do truth files need to be updated ("okified")?
      • after the reviewer has approved these changes, run okify_regtests to update the truth files
• if a JIRA ticket exists, make sure it is resolved properly

news fragment change types...

• changes/<PR#>.general.rst: infrastructure or miscellaneous change
• changes/<PR#>.docs.rst
• changes/<PR#>.stpipe.rst
• changes/<PR#>.datamodels.rst
• changes/<PR#>.scripts.rst
• changes/<PR#>.fits_generator.rst
• changes/<PR#>.set_telescope_pointing.rst
• changes/<PR#>.pipeline.rst

stage 1

• changes/<PR#>.group_scale.rst
• changes/<PR#>.dq_init.rst
• changes/<PR#>.emicorr.rst
• changes/<PR#>.saturation.rst
• changes/<PR#>.ipc.rst
• changes/<PR#>.firstframe.rst
• changes/<PR#>.lastframe.rst
• changes/<PR#>.reset.rst
• changes/<PR#>.superbias.rst
• changes/<PR#>.refpix.rst
• changes/<PR#>.linearity.rst
• changes/<PR#>.rscd.rst
• changes/<PR#>.persistence.rst
• changes/<PR#>.dark_current.rst
• changes/<PR#>.charge_migration.rst
• changes/<PR#>.jump.rst
• changes/<PR#>.clean_flicker_noise.rst
• changes/<PR#>.ramp_fitting.rst
• changes/<PR#>.gain_scale.rst

stage 2

• changes/<PR#>.assign_wcs.rst
• changes/<PR#>.badpix_selfcal.rst
• changes/<PR#>.msaflagopen.rst
• changes/<PR#>.nsclean.rst
• changes/<PR#>.imprint.rst
• changes/<PR#>.background.rst
• changes/<PR#>.extract_2d.rst
• changes/<PR#>.master_background.rst
• changes/<PR#>.wavecorr.rst
• changes/<PR#>.srctype.rst
• changes/<PR#>.straylight.rst
• changes/<PR#>.wfss_contam.rst
• changes/<PR#>.flatfield.rst
• changes/<PR#>.fringe.rst
• changes/<PR#>.pathloss.rst
• changes/<PR#>.barshadow.rst
• changes/<PR#>.photom.rst
• changes/<PR#>.pixel_replace.rst
• changes/<PR#>.resample_spec.rst
• changes/<PR#>.residual_fringe.rst
• changes/<PR#>.cube_build.rst
• changes/<PR#>.extract_1d.rst
• changes/<PR#>.resample.rst

stage 3

• changes/<PR#>.assign_mtwcs.rst
• changes/<PR#>.mrs_imatch.rst
• changes/<PR#>.tweakreg.rst
• changes/<PR#>.skymatch.rst
• changes/<PR#>.exp_to_source.rst
• changes/<PR#>.outlier_detection.rst
• changes/<PR#>.tso_photometry.rst
• changes/<PR#>.stack_refs.rst
• changes/<PR#>.align_refs.rst
• changes/<PR#>.klip.rst
• changes/<PR#>.spectral_leak.rst
• changes/<PR#>.source_catalog.rst
• changes/<PR#>.combine_1d.rst
• changes/<PR#>.ami.rst

other

• changes/<PR#>.wfs_combine.rst
• changes/<PR#>.white_light.rst
• changes/<PR#>.cube_skymatch.rst
• changes/<PR#>.engdb_tools.rst
• changes/<PR#>.guider_cds.rst

[melanieclarke]

Regression tests here, with locally generated truth files:
https://github.com/spacetelescope/RegressionTests/actions/runs/11296621434

Test failures are unrelated.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 61.85%. Comparing base (58429ee) to head (19f7809).
Report is 457 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8881      +/-   ##
==========================================
+ Coverage   61.83%   61.85%   +0.01%     
==========================================
  Files         377      377              
  Lines       38896    38877      -19     
==========================================
- Hits        24050    24046       -4     
+ Misses      14846    14831      -15     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[stscirij]

Looks like the chosen dataset only reads through 1 amp (NOUTPUTS=1), whereas the original used 4 and the title of the test implies that a 4 amp readout is desired?

[melanieclarke]

Looks like the chosen dataset only reads through 1 amp (NOUTPUTS=1), whereas the original used 4 and the title of the test implies that a 4 amp readout is desired?

Ah, thank you! I missed that. I'll look for another data set.

[melanieclarke]

Re-running regtests here with the new data set that has NOUTPUTS=4:
https://github.com/spacetelescope/RegressionTests/actions/runs/11446241421

Results look good; the 2 remaining failures are unrelated.

[melanieclarke]

@stscirij - I think this is ready for re-review.

[stscirij]

Looks good to me!