Skip to content

Structured outputs seems to be flaky #374

@8lank

Description

@8lank

After working for a while with the Claude Agent SDK, I've noticed some flakiness around structured outputs:

In many cases, when working with large data outputs, it seems that Claude is sometimes calling its internal tool StructuredOutput with a string that contains the object instead of the actual object. It then keeps looping to try to understand why the tool call is failing.

Example of StructuredOutput tool calls with different values for a same query:

Valid:

input={
  'description': "A 3-4 day family backpacking adventure ...",
  'plan': [
    {'title': 'Research and select trail options', ...},
    {'title': 'Apply for permits and finalize route', ...},
    ...
  ],
  'status': 'open',
  'summary': 'A transformative family backpacking journey...',
  'title': 'Pacific Northwest Family Backpacking Adventure'
}

Invalid:

input={
  'description': '{\n  "title": "Pacific Northwest Family Backpacking Adventure",\n  "summary": "...",\n  "description": "...",\n  "status": "open",\n  "plan": [ ... ]\n}'
}

Is this something that has been seen internally as well? Is there any plan to improve this?

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions