Allow background accumulation for combinations of full and subarray observations

[thomaswilliamsastro]

Closes #7807

This PR changes the way background accumulation is handled, to properly extract regions of the chip if combining e.g. main chip and subarray data. Tested on imaging data (works in these edge cases, and is identical to the old behaviour otherwise). Spectra should work but haven't explicitly been tested. I'm not at STScI so can't run the full regression suite.

Checklist for maintainers

• added entry in CHANGES.rst within the relevant release section
• updated or added relevant tests
• updated relevant documentation
• added relevant milestone
• added relevant label(s)
• ran regression tests, post a link to the Jenkins job below.
  How to run regression tests on a PR
• Make sure the JIRA ticket is resolved properly

[codecov]

Codecov Report

Patch coverage: 66.66% and project coverage change: -0.03% ⚠️

Comparison is base (9b22d5b) 76.39% compared to head (3bedeb6) 76.36%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7827      +/-   ##
==========================================
- Coverage   76.39%   76.36%   -0.03%     
==========================================
  Files         459      459              
  Lines       37074    37130      +56     
==========================================
+ Hits        28324    28356      +32     
- Misses       8750     8774      +24     

Flag	Coverage Δ		*Carryforward flag
nightly	77.36% <ø> (-0.02%)	⬇️	Carriedforward from f0fe4ce

*This pull request uses carry forward flags. Click here to find out more.

Files Changed	Coverage Δ
jwst/background/background_sub.py	81.21% <66.66%> (-12.37%)	⬇️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[hbushouse]

Looks good overall. Just a few comments and questions.

[hbushouse]

All the CI failures are due to a mismatch between jwst/master and the latest stdatamodels release. A new stdatamodels release is being created, which should fix this.

[thomaswilliamsastro]

Comments addressed, and I've rebased and pushed

[hbushouse]

Thanks for the updates. Looks good. Wouldn't hurt to add a short blurb to https://github.com/spacetelescope/jwst/blob/master/docs/jwst/background_step/description.rst to mention that the step now has the capability to create an average bkg image from inputs of different (overlapping) sizes.

Looks like the failing CI tests are due to numpy deprecation warnings from other code modules, but still need to check them all to confirm.

[hbushouse]

Started a regression test run at https://plwishmaster.stsci.edu:8081/job/RT/job/JWST-Developers-Pull-Requests/895

[hbushouse]

Oh wait, regarding the failing CI tests. It does look like it's encountering a problem in the new code in this PR when running one of the unit tests:
FAILED jwst/background/tests/test_background.py::test_nirspec_gwa - TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

which is due to:

        # Account for the fact these things are 1-indexed
        self.jmin = im.meta.subarray.ystart - 1
>       self.jmax = im.meta.subarray.ysize + self.jmin
E       TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

Suggests that ysize is None for whatever data is being used in the unit test. Perhaps the YSIZE keyword is just missing from that test data file.

[thomaswilliamsastro]

Could be, there's some code in there to catch if starts are missing but not sizes. Just made a quick edit on that which should hopefully catch it (unless subarray data, in which case it should likely error out anyway). Added a quick note in the description as well that this will use the overlapping regions

[hbushouse]

In addition to adding a trap in the code for missing xsize/ysize values, it looks like the unit test should also be updated so that it sets those to begin with in the fake images it's creating. It sets xstart/ystart, but not xsize/ysize. For example, lines could be added here https://github.com/spacetelescope/jwst/blob/master/jwst/background/tests/test_background.py#L46 and https://github.com/spacetelescope/jwst/blob/master/jwst/background/tests/test_background.py#L72 to set meta.subarray.xsize and ysize by either hardwiring to the same 10x10 dimensions used to create the datamodel or set to image.data.shape[-1] and image.data.shape[-2] just to leave the code more general.

[thomaswilliamsastro]

Done! I used the image shape in case these things change in the future

[hbushouse]

CI failures are unrelated.

[hbushouse]

Everything looks good now.