JP-3776: Improve AmiAnalyze memory usage

[emolter]

Resolves JP-3776

Closes #8860

This PR refactors the AmiAnalyze step to avoid creating a long list of 3-D arrays, reducing memory usage by a factor of ~13 for my test dataset (jw01349-c1007_20250319t131303_ami3_00002_asn.json). See ticket for memory usage flamegraphs - the one for this PR branch is named memray-flamegraph-output_refactor.html.

A beneficial side-effect of this refactor is that it removes a code pattern in which RawOifits objects required FringeFitter object as input, but some methods of FringeFitter required a RawOifits that was being initialized with that very same FringeFitter object, i.e. RawOifits(self). Now both classes require only an instrument data object, which I feel makes more logical sense and more cleanly splits up the roles of RawOifits and FringeFitter.

Memory usage tests to prevent a regression will be added by #9433

Tasks

• If you have a specific reviewer in mind, tag them.
• add a build milestone, i.e. Build 12.0 (use the latest build if not sure)
• Does this PR change user-facing code / API? (if not, label with no-changelog-entry-needed)
  • write news fragment(s) in changes/: echo "changed something" > changes/<PR#>.<changetype>.rst (see changelog readme for instructions)
  • update or add relevant tests
  • update relevant docstrings and / or docs/ page
  • start a regression test and include a link to the running job (click here for instructions)
    • Do truth files need to be updated ("okified")?
      • after the reviewer has approved these changes, run okify_regtests to update the truth files
• if a JIRA ticket exists, make sure it is resolved properly

[emolter]

regression tests started here https://github.com/spacetelescope/RegressionTests/actions/runs/14893626000

edit: Single failure was from JP-3941 and is not related

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 77.85%. Comparing base (b4f10ec) to head (16abf3f).
⚠️ Report is 678 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #9442   +/-   ##
=======================================
  Coverage   77.85%   77.85%           
=======================================
  Files         362      362           
  Lines       36287    36287           
=======================================
  Hits        28252    28252           
  Misses       8035     8035           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[drlaw1558]

LG2M; confirmed with some 1600-integration AMI data that the peak memory usage drops from 27 GB to < 1 GB. Output data look to be unchanged, as is the runtime.

[melanieclarke]

This looks sensible to me. It needs rebasing, and we should run the regtests again. They should now show the memory improvements, since #9433 was merged, right?

Do you want to wait for Rachel to review?

[emolter]

Do you want to wait for Rachel to review?

I don't think it's necessary to do so. There are a few "API changes" if someone were using these classes outside the pipeline, but my understanding from my last conversations with Rachel is that, while the team wants to maintain the flexibility of these classes for testing (i.e., don't just remove everything not used by pipeline) they don't have any active/established workflows that require these. And the refactor doesn't remove any information that isn't accessible via a straightforward workaround.

If it turns out they're unhappy with the changes we'd hear about it in testing anyway.

[emolter]

regtests https://github.com/spacetelescope/RegressionTests/actions/runs/14935304640

[emolter]

regression tests are clean. The memory usage tracker shows

test case | previous status | current status | previous peakmem | current peakmem
jwst.regtest.test_niriss_ami3::test_log_tracked_resources_ami3 |   |   | 1259MB | 161MB