add support for tensor learning rate (vs scalar)

[NirSonnenschein]

This change is intended to help enable support for using a tensor learning rate value vs a scalar ones.
We found this helpful in cases where the optimizer is torch.compiled (in such cases changing the scalar LR value could cause recompilation degrading the performance).
The implementation allows the model script to determine the type of LR value used by setting the initial value.

[NirSonnenschein]

Thanks @sfc-gh-truwase
small question: the CI failure doesn't seem to be related to the commit:
FAILED tests/unit/v1/zero/test_zero.py::TestZero3RepeatForwardLoop::test[True] - AttributeError: 'int' object has no attribute 'pt_reserved_cores_perc'
is this a known issue?

[eternalNight]

Thanks @sfc-gh-truwase small question: the CI failure doesn't seem to be related to the commit: FAILED tests/unit/v1/zero/test_zero.py::TestZero3RepeatForwardLoop::test[True] - AttributeError: 'int' object has no attribute 'pt_reserved_cores_perc' is this a known issue?

#7634 attempts to fix that, but is blocked because the CI seems not testing the right branch (yet).