[Bug]: logprobs not included in suggestion response

[allen-cook]

What happened?

I'm having trouble accessing logprobs in my chat suggestion response using Qwen Coder 2.5, the logprobs are being returned from the server successfully because I can access them under the original_response attribute.

acompletion({
  "custom_llm_provider": "text-completion-openai",
  "model": "qwen2p5-coder-7b",
  "api_base": YOUR_API_BASE
  "messages": [{"content": "hello", "role": "user"}],
  "stream": False,
  "logprobs": 1
}

Does not work: (logprobs doesn't exist)

suggestion.choices[0].logprobs.token_logprobs[0]

Does work:

suggestion._hidden_params['original_response']['choices'][0]['logprobs'].token_logprobs[0]

Relevant log output

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.55.9

Twitter / LinkedIn details

No response

[krrishdholakia]

hi @allen-cook can you share the raw response (either the original response attribute or the output in logs via litellm._turn_on_debug()

it would help us identify the gap in translation

[allen-cook]

hi @allen-cook can you share the raw response (either the original response attribute or the output in logs via litellm._turn_on_debug()

it would help us identify the gap in translation

Here's an example output from the OpenAI compatible provider we're using

{
  "id": "e73fc81b-5958-4199-b75d-566abec6c0ac",
  "object": "text_completion",
  "created": 1737653517,
  "model": "accounts/fireworks/models/qwen2p5-coder-7b",
  "choices": [
    {
      "index": 0,
      "text": "",
      "logprobs": {
        "tokens": [
          ""
        ],
        "token_logprobs": [
          -1.16139472
        ],
        "top_logprobs": [
          {
            "<|endoftext|>": -1.16139472
          }
        ],
        "text_offset": [
          0
        ],
        "token_ids": [
          151643
        ]
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 44,
    "total_tokens": 45,
    "completion_tokens": 1
  }
}

[krrishdholakia]

from the OpenAI compatible provider

@allen-cook fyi - we have native support for fireworks ai - https://docs.litellm.ai/docs/providers/fireworks_ai

[ishaan-jaff]

@allen-cook is this resolved ?

[allen-cook]

@allen-cook is this resolved ?

Yes, when using latest this is fixed, When this is released I will update our gem

[krrishdholakia]

This is already live since v1.60.2-dev1 - c8494ab