memory and runtime benchmark/testing infrastructure

[stscijgbot-jp]

Issue JP-3775 was created on JIRA by Ned Molter:

Add and implement tools to ensure that pull requests do not inadvertently cause significant regressions in memory usage or runtime.

At a hack day discussion, we targeted initial implementation and testing in stcal, since it's a smaller repository but still has non-trivial memory- and runtime-hungry functions (e.g., outlier detection median calculation).  Goal is very quick adoption by JWST, and adoption by Roman once other outstanding regression test issues have been resolved.

The preferred workflow would be to have decorators that could be added to tests to flag them as memory- and/or runtime-checked.  At time of writing, codspeed is being considered for runtime, and pytest-memray is being considered for memory.  

Task list for now:

• Add memory usage unit tests for methods where memory optimization has been attempted, e.g. the median calculator in outlier detection.
• Test out codspeed by starting to use it in stcal
• Figure out a good general way to report memory usage on PR branches
• Decide whether memory usage regression tests are needed, or if benchmarking and showing the results to PR developers is good enough

[stscijgbot-jp]

Comment by Ned Molter on JIRA:

A PR in stcal adds a context manager that makes it easier to compare the peak memory usage of a code block against an expected usage.  A single unit test to the median calculator is implemented that uses the context manager, and this test should be used as an example of how to implement a test where the expected memory usage can be calculated straightforwardly based on the input size.  The new context manager can be imported into roman and jwst.

PR link here: spacetelescope/stcal#299 

edit: this part of the work is complete

[stscijgbot-jp]

Comment by Tyler Pauly on JIRA:

Implementation of memory usage tracking in RTs was completed in #9357 and #9433