[Feature]: (non deprecated) boto3 support or awsume interoperability

[jonha892]

The Feature

awsume is a library/tool to authenticate with the AWS. It has some caching built in such that users only have to type in MFA codes once per day.

Used as a library, it provides this functionality to boto3 clients (https://awsu.me/advanced/non-interactive-awsume.html).

This way developers don't have to provide new MFA codes on every program execution, which comes in handy during local testing.

I'd like this functionality in litellm. The simplest/most flexible way for me as a user would be to fix the usage of boto3-bedrock-client. Currently this is deprecated (https://docs.litellm.ai/docs/providers/bedrock#passing-an-external-bedrockruntimeclient-as-a-parameter---completion).

Motivation, pitch

Huge initial productivity boost when developing with aws bedrock.

Many organizations use dedicated aws profiles, secured by MFA.
Using awsume is one popular and convinient way to locally access aws resources in professional settings.

Are you a ML Ops Team?

No

Twitter / LinkedIn details

No response

[ishaan-jaff]

@jonha892 can you clarify, what is the problem you are trying to solve?

We might not be able to add the new dependency - but why do you want it ? Can add more details ?

[jonha892]

Sure, it might just be very specific to our use-case.

In enterprise environments we use different AWS profiles, such that local development would look like this:

import boto3

session = boto3.Session(profile_name='dev')
client = session.client('bedrock-runtime')

Thats a bit of a pain since the credentials are secured by a MFA device. Boto3 doesn't use caching behind the scenes (afaik) such that we'd have to do the MFA authentication every time we execute the code (locally). Same goes for AWS cli commands or IAC deployments.

One way to optimize local development is to use awsume (or aws-vault, or some other alternative). We found awsume to be particularly pleasent to work with. It uses a session/refresh tokens such that MFA flows only have to be done daily.
It integrates into python like this:

from awsume.awsumepy import awsume

session = awsume('dev')
client = session.client('bedrock-runtime')

So in this specific scenario it would be really handy if we could just hand a bedrock-runtime instance to litellm where we do the authentication ourselves.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.