feat: allow mixed precision policy as dtype

[kmehant]

What does this PR do?

Allow mixed precision to be passed as a dtype string from accelerate cli flag or fsdp_config in accelerate config file.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@S1ro1 @SunMarc

[kmehant]

@S1ro1 WDYT on this?

[S1ro1]

Hey @kmehant currently OOO, will give it a look next week!

[kmehant]

Sure, @S1ro1 thanks!

[muellerzr]

IMO: this simplifies quite a few things without over-bloating params too much. I think it's an easy win, however I'll leave this to @S1ro1 and @SunMarc and go back to the leave-grave for now :)

[S1ro1]

LGTM. Can we maybe add tests? (should be simple enough to just adapt the current ones for MP to also test this)

[SunMarc]

Nice, just add a test if possible

[kmehant]

@S1ro1 @SunMarc

Extended the test case and passes!

I have removed the flag part of the PR since, I see mixed_precision accelerate argument is already taking care of it on setting the mixed precision for FSDP1/2.

[S1ro1]

Thank you! LGTM

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.