Tool- and domain-specific telemetry for MCP server

[devin-ai-integration]

Summary

This issue proposes enhancing the MCP server telemetry to track more granular information about MCP tool usage and domain-specific activity.

Background

The current telemetry module (airbyte/_util/telemetry.py) tracks MCP mode via a boolean flag in the flags dictionary:

def get_env_flags() -> dict[str, Any]:
    flags: dict[str, bool | str] = {
        "CI": meta.is_ci(),
        "LANGCHAIN": meta.is_langchain(),
        "MCP": meta.is_mcp_mode(),  # Current MCP tracking
        "NOTEBOOK_RUNTIME": ...
    }

This allows us to filter events by MCP usage, but doesn't provide insight into which specific MCP tools are being used or how they're being used.

Proposed Enhancements

1. Track MCP Tool Invocations

Add telemetry for which MCP tools are being called:

• list_connectors, get_connector_info, get_api_docs_urls (registry domain)
• validate_connector_config, list_source_streams, sync_source_to_cache, run_sql_query (local domain)
• deploy_source_to_cloud, create_connection_on_cloud, run_cloud_sync (cloud domain)

2. Track MCP Tool Success/Failure Rates

For each tool invocation, track:

• Tool name
• Domain (registry, local, cloud)
• Success/failure status
• Error type (if failed)

3. Track MCP Session Metrics

• Session duration
• Number of tool calls per session
• Tool call sequences (for understanding common workflows)

Implementation Considerations

• Privacy: Continue to hash any potentially identifying information
• Performance: Use async/non-blocking telemetry calls
• Opt-out: Respect existing DO_NOT_TRACK environment variable

Related

• PR for MCP telemetry reports in airbyte-dataops-poc-internal: https://github.com/airbytehq/airbyte-dataops-poc-internal/pull/14
• MCP server implementation: airbyte/mcp/
• Telemetry module: airbyte/_util/telemetry.py

---

Requested by AJ Steers (@aaronsteers)