[Bug]: Invalid metadata when calling OpenAI models

[wipash]

What happened?

When calling OpenAI models, I get this error:

Error occurred while generating model response. Please try again. Error: Error: 400 litellm.BadRequestError: OpenAIException - Error code: 400 - {'error': {'message': "Invalid 'metadata': too many properties. Expected an object with at most 16 properties, but got an object with 28 properties instead.", 'type': 'invalid_request_error', 'param': 'metadata', 'code': 'object_above_max_properties'}} Received Model Group=gpt-4o-mini Available Model Group Fallbacks=None

This happens with gpt-4o, gpt-4o-mini, and o3-mini
I have the same models configured via Azure OpenAI, and they work fine.

I can trigger this error from the LiteLLM playground:

Config:

model_list:
  # OpenAI
  - model_name: o3-mini
    litellm_params:
      model: openai/o3-mini
      api_key: os.environ/OPENAI_API_KEY

  - model_name: gpt-4o
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY

  - model_name: gpt-4o-mini
    litellm_params:
      model: openai/gpt-4o-mini
      api_key: os.environ/OPENAI_API_KEY

  # Azure
  - model_name: azure-gpt-4o-mini
    litellm_params:
      model: azure/gpt-4o-mini
      api_key: os.environ/AZURE_OPENAI_AUSTRALIAEAST_API_KEY
      api_base: os.environ/AZURE_OPENAI_AUSTRALIAEAST_ENDPOINT
      api_version: 2024-08-01-preview

  - model_name: azure-gpt-4o
    litellm_params:
      model: azure/gpt-4o
      api_key: os.environ/AZURE_OPENAI_WESTUS_API_KEY
      api_base: os.environ/AZURE_OPENAI_WESTUS_ENDPOINT
      api_version: 2024-08-01-preview

  - model_name: azure-o1
    litellm_params:
      model: azure/o1
      api_key: os.environ/AZURE_OPENAI_EASTUS2_API_KEY
      api_base: os.environ/AZURE_OPENAI_EASTUS2_ENDPOINT
      api_version: 2024-12-01-preview

  - model_name: azure-o3-mini
    litellm_params:
      model: azure/o3-mini
      api_key: os.environ/AZURE_OPENAI_EASTUS2_API_KEY
      api_base: os.environ/AZURE_OPENAI_EASTUS2_ENDPOINT
      api_version: 2024-12-01-preview


general_settings:
  proxy_batch_write_at: 60
  database_connection_pool_limit: 10

  disable_spend_logs: false
  disable_error_logs: false

  background_health_checks: false
  health_check_interval: 300

  store_model_in_db: true

litellm_settings:
  request_timeout: 600
  json_logs: true
  enable_preview_features: true
  redact_user_api_key_info: true
  turn_off_message_logging: true

Relevant log output

{
  "message": "Inside async function with retries: args - (); kwargs - {'user': 'redacted', 'proxy_server_request': {'url': 'http://litellm.litellm.svc.cluster.local:4000/chat/completions', 'method': 'POST', 'headers': {'content-length': '164', 'accept': 'application/json', 'content-type': 'application/json', 'user-agent': 'OpenAI/JS 4.71.1', 'x-stainless-lang': 'js', 'x-stainless-package-version': '4.71.1', 'x-stainless-os': 'Linux', 'x-stainless-arch': 'x64', 'x-stainless-runtime': 'node', 'x-stainless-runtime-version': 'v20.18.2', 'x-stainless-helper-method': 'stream', 'x-stainless-retry-count': '0', 'accept-encoding': 'gzip,deflate', 'host': 'litellm.litellm.svc.cluster.local:4000', 'connection': 'keep-alive'}, 'body': {'model': 'o3-mini', 'user': '6736c195266216764eecdf43', 'stream': True, 'messages': [{'role': 'user', 'content': 'Hi there!'}]}}, 'metadata': {'requester_metadata': {}, 'user_api_key_hash': 'redacted', 'user_api_key_alias': 'redacted', 'user_api_key_team_id': 'redacted', 'user_api_key_user_id': None, 'user_api_key_org_id': None, 'user_api_key_team_alias': 'redacted', 'user_api_key_end_user_id': 'redacted', 'user_api_key': 'redacted', 'user_api_end_user_max_budget': None, 'litellm_api_version': '1.60.0', 'global_max_parallel_requests': None, 'user_api_key_team_max_budget': None, 'user_api_key_team_spend': 0.001455905, 'user_api_key_spend': 0.001455905, 'user_api_key_max_budget': None, 'user_api_key_model_max_budget': {}, 'user_api_key_metadata': {'service_account_id': 'redacted'}, 'headers': {'content-length': '164', 'accept': 'application/json', 'content-type': 'application/json', 'user-agent': 'OpenAI/JS 4.71.1', 'x-stainless-lang': 'js', 'x-stainless-package-version': '4.71.1', 'x-stainless-os': 'Linux', 'x-stainless-arch': 'x64', 'x-stainless-runtime': 'node', 'x-stainless-runtime-version': 'v20.18.2', 'x-stainless-helper-method': 'stream', 'x-stainless-retry-count': '0', 'accept-encoding': 'gzip,deflate', 'host': 'litellm.litellm.svc.cluster.local:4000', 'connection': 'keep-alive'}, 'endpoint': 'http://litellm.litellm.svc.cluster.local:4000/chat/completions', 'litellm_parent_otel_span': None, 'requester_ip_address': '', 'model_group': 'o3-mini'}, 'litellm_call_id': '8e10b1f1-f07c-47e4-bcdf-813d1bb22cb5', 'litellm_logging_obj': <litellm.litellm_core_utils.litellm_logging.Logging object at 0x7f3701f4d400>, 'model': 'o3-mini', 'messages': [{'role': 'user', 'content': 'Hi there!'}], 'stream': True, 'original_function': <bound method Router._acompletion of <litellm.router.Router object at 0x7f3701f4fa10>>, 'num_retries': 2, 'litellm_trace_id': '5757f792-6509-4fb9-a626-069fdc20bca8'}",
  "level": "DEBUG",
  "timestamp": "2025-02-03T22:42:09.845082"
}

I have these environment variables configured:

TZ: Pacific/Auckland
LITELLM_LOG: "DEBUG"
LITELLM_MODE: "PRODUCTION" # This disabled loading .env env vars
LITELLM_DONT_SHOW_FEEDBACK_BOX: "true"
NO_DOCS: "true"

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.60.0

Twitter / LinkedIn details

No response

[wipash]

@ishaan-jaff any idea on this? I still haven't seen any other reports of it so maybe it's a problem on my end, but my config is so simple that I can't work out how it would be.

[jruokola]

If you're running docker test with the new litellm-database image

[ishaan-jaff]

the chat ui works for me, perhaps it's something on your config

[wipash]

I did some more experimenting, and it seems to be caused by the config option enable_preview_features: true. When that's set to true I get the error on OpenAI API models, when commenting it out the OpenAI API models work again.

So I guess it's happening because of this?

litellm/litellm/main.py

Lines 1682 to 1685 in 6a4013a

	if (
	litellm.enable_preview_features and metadata is not None
	): # [PREVIEW] allow metadata to be passed to OPENAI
	optional_params["metadata"] = add_openai_metadata(metadata)

I don't have a dev environment set up for litellm yet so can't debug it, but will try to find some time this week.

[krrishdholakia]

This is caused by enable_preview_features:true

The metadata Param is sent to openai but this currently contains non-openai metadata as well (liteLLM info) causing OpenAI to throw an error.

[krrishdholakia]

This is now fixed - please update to main latest

[rossm-mf]

I seemed to have the same issue ... almost. my setup is openwebui going throught litellm and I add metadata through this https://github.com/thiswillbeyourgithub/openwebui_custom_pipes_filters/blob/main/filters/add_metadata.py

I did opened an issue with the filter I user here : thiswillbeyourgithub/openwebui_custom_pipes_filters#3

but it does seem to crash the same way as here and only for openai/gpts models.

17:41:09 - LiteLLM Proxy:DEBUG: proxy_server.py:3718 - An error occurred: litellm.BadRequestError: AzureException BadRequestError - Error code: 400 - {'error': {'message': "Invalid 'metadata': too many properties. Expected an object with at most 16 properties, but got an object with 24 properties instead.", 'type': 'invalid_request_error', 'param': 'metadata', 'code': 'object_above_max_properties'}}No fallback model group found for original model_group=azure/gpt-4o-mini. Fallbacks=[{'onprem/casperhansen/llama-3-70b-instruct-awq': ['gemini/gemini-2.0-flash']}]. Received Model Group=azure/gpt-4o-mini
Available Model Group Fallbacks=None
Error doing the fallback: litellm.BadRequestError: AzureException BadRequestError - Error code: 400 - {'error': {'message': "Invalid 'metadata': too many properties. Expected an object with at most 16 properties, but got an object with 24 properties instead.", 'type': 'invalid_request_error', 'param': 'metadata', 'code': 'object_above_max_properties'}}No fallback model group found for original model_group=azure/gpt-4o-mini. Fallbacks=[{'onprem/casperhansen/llama-3-70b-instruct-awq': ['gemini/gemini-2.0-flash']}] LiteLLM Retried: 1 times, LiteLLM Max Retries: 2 LiteLLM Retried: 1 times, LiteLLM Max Retries: 2 

Key Name: `####-staging`
Team: `####`
Model: azure/gpt-4o-mini
API Base: `https://###########.openai.azure.com/`
Messages: `[{'role': 'user', 'content': 'say one word'}]`
model_group: `azure/gpt-4o-mini`

deployment: `azure/gpt-4o-mini`


 Debug this by setting `--debug`, e.g. `litellm --model gpt-3.5-turbo --debug`

[krrishdholakia]

Hi @rossm-mf do you also do enable_preview_features:true under litellm settings?

If not - please share your config for repro

[rossm-mf]

I forced the value to enable_preview_features:false but I wasn't using it before anyway. I also tried using drop_params it was recommended by the addMetadata author.

So far doesn't help

model_list:
  - model_name: azure/gpt-4o-mini
    litellm_params:
      model: azure/gpt-4o-mini
      api_base: os.environ/AZURE_API_BASE
      api_key: os.environ/AZURE_API_KEY
      api_version: "2024-08-01-preview"
      drop_params: True
  - model_name: openai/gpt-4o-mini
    litellm_params: 
      model: openai/gpt-4o-mini
      api_key: os.environ/OPENAI_API_KEYS
      drop_params: True

general_settings:
  background_health_checks: False
  store_model_in_db: True
  # Only USE when running LiteLLM on your VPC. Allow requests to still be processed even if the DB is unavailable. We recommend doing this if you're running LiteLLM on VPC that cannot be accessed from the public internet.
  allow_requests_on_db_unavailable: True 

litellm_settings:
  drop_params: True
  # Switch off Debug Logging, ensure your logs do not have any debugging on
  set_verbose: True
  json_logs: True
  enable_preview_features: False
  turn_off_message_logging: True
  # set cache responses to True, litellm defaults to using a redis cache, NEED ONE if enabled

[krrishdholakia]

Hi @rossm-mf when you enable --detailed_debug you should see the raw request received by litellm, can you share that?

I can try to repro this as well.