Introduce Thinking Config Across Anthropic and OpenAI Providers

[ksylvan]

Introduce Thinking Config Across Anthropic and OpenAI Providers

Summary

Introduce a provider-agnostic thinking/reasoning control across Fabric with a new CLI flag: --thinking. This enables users to request different levels of reasoning from Anthropic and OpenAI models in a consistent way. The change includes:

• New --thinking flag and shell completions.
• Domain support for thinking levels and standardized token budgets.
• Anthropic: map thinking levels (and numeric tokens) to ThinkingConfig.
• OpenAI: map thinking levels to Reasoning Effort for both Chat Completions and Responses APIs.
• Dry-run output shows selected thinking level.
• Overhaul of suggest_pattern docs: categorized pattern index, workflows, and usage guidance.
• Minor VSCode dictionary updates.
• Add changelog entry.

Related Issues

Closes #1694

Files Changed

• .vscode/settings.json

  • Added dictionary entries: Anki, DMARC, wireframes.

• cmd/generate_changelog/incoming/1700.txt

  • New changelog entry summarizing this PR.

• completions/_fabric (zsh)

  • Added --thinking completion with allowed values: off, low, medium, high.

• completions/fabric.bash

  • Included --thinking in options and tab-completion choices.

• completions/fabric.fish

  • Added --thinking with completion choices.

• data/patterns/suggest_pattern/system.md

  • Rewrote and expanded the system prompt: clear identity/purpose, stepwise approach, categorized pattern catalog, workflow guidance, and output instructions.

• data/patterns/suggest_pattern/user.md

  • Major restructure: patterns organized by category (AI, ANALYSIS, DEVELOPMENT, SECURITY, etc.), brief descriptions, and usage guidance.

• data/patterns/suggest_pattern/user_clean.md

  • Deleted (unused duplicate).

• data/patterns/suggest_pattern/user_updated.md

  • Deleted (replaced by consolidated user.md).

• internal/cli/flags.go

  • Added Flags.Thinking domain.ThinkingLevel with help text.
  • Propagated Thinking to ChatOptions in BuildChatOptions.

• internal/cli/flags_test.go

  • Updated expected ChatOptions with default Thinking value.

• internal/domain/domain.go

  • Added Thinking field to ChatOptions.

• internal/domain/thinking.go

  • New: ThinkingLevel type and constants (off, low, medium, high).
  • Standardized token budgets for Anthropic thinking-enabled models:
    • low: 1024, medium: 2048, high: 4096.

• internal/plugins/ai/anthropic/anthropic.go

  • Added parseThinking() to translate CLI thinking level (or numeric tokens) into anthropic.ThinkingConfigParamUnion.
  • Inject Thinking config into message params when specified.

• internal/plugins/ai/dryrun/dryrun.go

  • Show Thinking level in dry-run formatted options.

• internal/plugins/ai/openai/chat_completions.go

  • Map Thinking level to ReasoningEffort for Chat Completions requests.

• internal/plugins/ai/openai/openai.go

  • Added parseReasoningEffort() mapping low/medium/high to shared.ReasoningEffort.
  • Set ReasoningEffort for Chat Completions and Reasoning.Effort for Responses API when specified.

• internal/server/chat.go

  • Thread Thinking from API request to ChatOptions.

Code Changes

• CLI flag definition:

  // internal/cli/flags.go
  Thinking domain.ThinkingLevel `long:"thinking" yaml:"thinking" description:"Set reasoning/thinking level (e.g., off, low, medium, high, or numeric tokens for Anthropic)"`

• Domain support:

  // internal/domain/thinking.go
  type ThinkingLevel string
  
  const (
    ThinkingOff    ThinkingLevel = "off"
    ThinkingLow    ThinkingLevel = "low"
    ThinkingMedium ThinkingLevel = "medium"
    ThinkingHigh   ThinkingLevel = "high"
  )
  
  var ThinkingBudgets = map[ThinkingLevel]int64{
    ThinkingLow:    1024,
    ThinkingMedium: 2048,
    ThinkingHigh:   4096,
  }

• Anthropic mapping:

  // internal/plugins/ai/anthropic/anthropic.go
  func parseThinking(level domain.ThinkingLevel) (anthropic.ThinkingConfigParamUnion, bool) {
    lower := strings.ToLower(string(level))
    switch domain.ThinkingLevel(lower) {
    case domain.ThinkingOff:
      disabled := anthropic.NewThinkingConfigDisabledParam()
      return anthropic.ThinkingConfigParamUnion{OfDisabled: &disabled}, true
    case domain.ThinkingLow, domain.ThinkingMedium, domain.ThinkingHigh:
      if budget, ok := domain.ThinkingBudgets[domain.ThinkingLevel(lower)]; ok {
        return anthropic.ThinkingConfigParamOfEnabled(budget), true
      }
    default:
      if tokens, err := strconv.ParseInt(lower, 10, 64); err == nil {
        return anthropic.ThinkingConfigParamOfEnabled(tokens), true
      }
    }
    return anthropic.ThinkingConfigParamUnion{}, false
  }

• OpenAI mapping:

  // internal/plugins/ai/openai/openai.go
  func parseReasoningEffort(level domain.ThinkingLevel) (shared.ReasoningEffort, bool) {
    switch domain.ThinkingLevel(strings.ToLower(string(level))) {
    case domain.ThinkingLow:
      return shared.ReasoningEffortLow, true
    case domain.ThinkingMedium:
      return shared.ReasoningEffortMedium, true
    case domain.ThinkingHigh:
      return shared.ReasoningEffortHigh, true
    default:
      return "", false
    }
  }

• Dry-run visibility:

  // internal/plugins/ai/dryrun/dryrun.go
  if opts.Thinking != "" {
    builder.WriteString(fmt.Sprintf("Thinking: %s\n", string(opts.Thinking)))
  }

Reason for Changes

• Add a simple, consistent user-facing control for model reasoning/thinking across providers.
• Standardize Anthropic thinking budgets to common levels and allow advanced users to specify a numeric token budget.
• Surface OpenAI’s Reasoning Effort in a consistent way with the same flag.
• Improve discoverability and usability via shell completions and dry-run visibility.
• Make suggest_pattern documentation substantially more useful with categorized patterns, workflows, and practical examples.

Impact of Changes

• User experience: New --thinking flag offers a straightforward way to request more/less reasoning. Defaults remain unchanged; no behavior change unless the flag is used.
• Anthropic: Sets ThinkingConfig when present. Levels map to sensible defaults; advanced numeric override supported.
• OpenAI: Sets Reasoning Effort on supported models via both Chat Completions and Responses API paths. Ignored if model doesn’t support reasoning effort.
• Dry-run: Users can verify reasoning level before sending a request.
• Docs: Suggest pattern becomes an entry point for discovering patterns by category and planning multi-step workflows.

Test Plan

• Unit tests:

  • Updated flags_test.go to include default Thinking value; run go test ./... to ensure compilation and test pass.

• Manual smoke tests:

  • Dry run shows thinking level:
    • fabric -p ai --thinking high --dry-run
    • Confirm “Thinking: high” in formatted options.
  • Anthropic reasoning:
    • fabric -V anthropic -m claude-3-7-sonnet-thought -p ai --thinking medium
    • fabric -V anthropic -m claude-3-7-sonnet-thought -p ai --thinking 8192
    • Verify successful responses; compare cost/latency vs. off/low.
  • OpenAI reasoning:
    • Responses API (default): fabric -V openai -m o4-mini -p ai --thinking high
    • Chat Completions API: fabric -V openai -m o4-mini -p ai --thinking low --disable-responses-api
    • Confirm responses are returned and no client errors for supported models.

• Shell completions:

  • zsh/bash/fish: verify --thinking offers choices off/low/medium/high.

• Docs:

  • Ensure suggest_pattern loads and renders; internal links or examples are coherent.

Additional Notes

• The --thinking completions offer off/low/medium/high; numeric token inputs (Anthropic-only advanced usage) are accepted by the flag even though not suggested by completion.
• OpenAI Reasoning Effort is set only when a supported model is used. For unsupported models, the SDK should ignore the field; if not, we may need conditional gating by model name.
• REST API:
  • Server now forwards request.Thinking to ChatOptions. Ensure the client request DTO includes a Thinking field; otherwise, add it to maintain parity with CLI usage.
• Interactions with suppress-think:
  • --suppress-think still controls visibility of … content in output; --thinking controls whether/what kind of reasoning is requested from the provider. They are complementary.
• Backward compatibility:
  • No default behavior changes; the feature is opt-in.