JP-3438: Allow custom grouping of models

[mcara]

Resolves JP-3438
Closes #7996

This PR adds support for custom group ID so that users can conveniently modify how input models are grouped for the tweakreg and skymatch steps.

Checklist for maintainers

• added entry in CHANGES.rst within the relevant release section
• updated or added relevant tests
• updated relevant documentation
• added relevant milestone
• added relevant label(s)
• ran regression tests, post a link to the Jenkins job below.
  How to run regression tests on a PR
• Make sure the JIRA ticket is resolved properly

[codecov]

Codecov Report

All modified lines are covered by tests ✅

Files	Coverage Δ
jwst/datamodels/container.py	82.56% <100.00%> (+0.84%)	⬆️

📢 Thoughts on this report? Let us know!.

[nden]

Can you add a test?
If I understand this correctly I should be able to edit an association file and add a custom group_id. This does not work. Is group_id even saved to an association_file?

[hbushouse]

Will need updates to RTD somewhere (wherever appropriate) to explain this new capability.

[mcara]

Can you add a test? If I understand this correctly I should be able to edit an association file and add a custom group_id. This does not work. Is group_id even saved to an association_file?

It does work. Can you explain what is not working as expected?

I added a unit test. I had to modify the headers.fits file to add the following meta info:

meta.observation.sequence_id = '2'
meta.observation.visit_group = '02'
meta.observation.activity_id = '01'

Without sequence_id , visit_group, and activity_id defined, ModelContainer.models_grouped would not work.

[nden]

Can you explain what is not working as expected?

I was trying to generate an association file with a custom group_id . As described somewhere above I opened the files, assigned model.meta.goup_id and saved them. Then I ran asn_from_list to generate the association file. The new association is missing goup_id. This may be a user error or an association enhancement.
However, it looks like the only way to use a custom group_id or not group observations is to edit an association. Does it make sense to have a parameter to calweb_image3, tweakreg or sky_match which disables grouping?

[mcara]

I was trying to generate an association file with a custom group_id . As described somewhere above I opened the files, assigned model.meta.goup_id and saved them. Then I ran asn_from_list to generate the association file.

asn_from_list would need to be updated in order to be able to do this. Currently, this is not supported and I am not sure there is need for this: If you edited the files and set model.group_id (NOT model.meta.goup_id) then ModelContainer will pick this up. This PR is similar to #7022 in how it handles stuff. In fact, most machinery to support reading extra members from ASN and setting corresponding model attributes was done in that PR.

However, it looks like the only way to use a custom group_id or not group observations is to edit an association.

If you set model.group_id and save the model, it should work.

Does it make sense to have a parameter to calweb_image3, tweakreg or sky_match which disables grouping?

In principle yes, but it is harder to achieve that, I think... And it makes it easier to have confusing situations like what takes precedence. I need to investigate this a little.

[nden]

This looks good to me. The other enhancements can be done in a different PR. Need to rebase this one.