-
Notifications
You must be signed in to change notification settings - Fork 31
feat: Add mcp transport protocol #345
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
114 commits
Select commit
Hold shift + click to select a range
04f7962
add basic code
twishabansal 9ba5bbc
fixes
twishabansal 34cb50a
test fix
twishabansal ea417ba
new unit tests
twishabansal aa356b1
Merge branch 'main' into mcp-restructure
twishabansal 7c0f917
rename ToolboxTransport
twishabansal 434e91d
add py3.9 support
twishabansal 31d0f9e
fix langchain tool tests
twishabansal b50513b
test fix
twishabansal c222d25
lint
twishabansal b244740
fix tests
twishabansal 4b0f561
Merge branch 'main' into mcp-restructure
twishabansal d93f4dd
move manage session into transport
twishabansal f9d8e64
move warning to diff file
twishabansal d6361ed
avoid code duplication
twishabansal 0f3bacc
fix tests
twishabansal 0a79f5a
lint
twishabansal fcf7da3
remove redundant tests
twishabansal c65a94a
make invoke method return str
twishabansal e2269f8
lint
twishabansal 25802e2
fix return type
twishabansal d41aed8
small refactor
twishabansal 04263ff
refactor: remove transport logic from client tests
twishabansal 47a2a46
try
twishabansal 9cd9a79
version negotiation
twishabansal 5964bcc
small changes
twishabansal 4bac725
lint
twishabansal d8c6efb
fix endpoint
twishabansal b17e3ee
add some todos
twishabansal 5b4d12c
lint
twishabansal 388c7f9
initialise in init
twishabansal ef4e543
lint
twishabansal da384be
add support for 'Mcp-session-id'
twishabansal c2ad274
lint
twishabansal e88dfa7
add todo
twishabansal c9728a9
add mcp protocol version to the latest protocol
twishabansal c66dd26
add test coverage
twishabansal 3cd00ea
small fix
twishabansal 11ac6a2
small fix
twishabansal 02baad7
small fix
twishabansal 6ae38e1
thread fixes
twishabansal fb59bb5
try
twishabansal 765db81
add tests
twishabansal f1c0807
lint
twishabansal 24db78d
change small
twishabansal dcc811a
nit
twishabansal a4a4f55
small debugging
twishabansal 19a1cf2
add todos
twishabansal 914ec46
small bug fixes
twishabansal e922472
add todo
twishabansal 8c14096
remove id field from notifications
twishabansal 6c97083
refactor
twishabansal 9dfa8cb
preprocess tools with empty params
twishabansal 6f74838
fix types
twishabansal 9118a89
fix bugs
twishabansal fbce7e9
better error log
twishabansal b6b2dbe
small cleanup
twishabansal ac2a924
handle notifications
twishabansal 1fd0581
fix unit tests
twishabansal ec17eb8
lint
twishabansal 1cffac1
decouple client from transport
twishabansal cc30a17
lint
twishabansal 2f04c95
use toolbox protocol for e2e tests
twishabansal d80c41f
add e2e tests for mcp
twishabansal baf9d06
lint
twishabansal cd9841e
remove mcp as default protocol
twishabansal 83030dc
remove auth tests from mcp
twishabansal bb8dc97
remove redundant lines
twishabansal 8920538
remove redundant lines
twishabansal f70710f
lint
twishabansal 80c688a
revert some changes
twishabansal 4c42d33
initialise session in a better way
twishabansal 9c119e8
small fix
twishabansal 3a181ab
added more test cov
twishabansal 67bbb24
lint
twishabansal 31809a1
rename private method
twishabansal e281556
Made methods private
twishabansal e0a1337
lint
twishabansal 85e5d29
rename base url
twishabansal ac2acfe
resolve comment
twishabansal d061f3e
better readability
twishabansal 09866c4
Merge branch 'main' into mcp-restructure
twishabansal c7455a8
Merge branch 'main' into mcp-restructure
twishabansal f43909e
fix tests
twishabansal f07d4b3
lint
twishabansal b0a8e45
Merge branch 'mcp-restructure' into client-transport-decouple
twishabansal 815efed
fix tests
twishabansal 999cd3b
lint
twishabansal 45d66d9
refactor mcp versions
twishabansal a444a4b
lint
twishabansal 073e930
added test coverage
twishabansal 1e2d09c
Merge branch 'client-transport-decouple' into mcp-transport-implement
twishabansal 05e7cdc
Merge branch 'main' into mcp-transport-implement
twishabansal d916093
refactor mcp
twishabansal ef15144
lint
twishabansal 672e5d9
improve cov
twishabansal 769f4a3
lint
twishabansal b814bed
removed process id
twishabansal 1dfb828
Update class name
twishabansal ae260d0
remove mcp latest
twishabansal a180458
rename mcp.py
twishabansal d29b3d5
have a single method for session init
twishabansal efc59fd
lint
twishabansal bc6da15
better type checks for v20241105
twishabansal fd0b9a5
Revert "better type checks for v20241105"
twishabansal 7388c7b
update type checking
twishabansal 6349293
lint
twishabansal 1fd63a0
clean file
twishabansal 064aaa7
refactor files
twishabansal c058b65
refactor all versions
twishabansal 030be42
fix mypy errors
twishabansal 63d8a5a
refactor properly
twishabansal 332ff09
lint
twishabansal 0943ed5
run mcp e2e tests on all versions
twishabansal File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
23 changes: 23 additions & 0 deletions
23
packages/toolbox-core/src/toolbox_core/mcp_transport/__init__.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| # Copyright 2025 Google LLC | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| from .v20241105.mcp import McpHttpTransportV20241105 | ||
| from .v20250326.mcp import McpHttpTransportV20250326 | ||
| from .v20250618.mcp import McpHttpTransportV20250618 | ||
|
|
||
| __all__ = [ | ||
| "McpHttpTransportV20241105", | ||
| "McpHttpTransportV20250326", | ||
| "McpHttpTransportV20250618", | ||
| ] |
102 changes: 102 additions & 0 deletions
102
packages/toolbox-core/src/toolbox_core/mcp_transport/transport_base.py
twishabansal marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,102 @@ | ||
| # Copyright 2025 Google LLC | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| import asyncio | ||
| from abc import ABC, abstractmethod | ||
| from typing import Optional | ||
|
|
||
| from aiohttp import ClientSession | ||
|
|
||
| from ..itransport import ITransport | ||
| from ..protocol import ( | ||
| AdditionalPropertiesSchema, | ||
| ParameterSchema, | ||
| Protocol, | ||
| ToolSchema, | ||
| ) | ||
|
|
||
|
|
||
| class _McpHttpTransportBase(ITransport, ABC): | ||
| """Base transport for MCP protocols.""" | ||
|
|
||
| def __init__( | ||
| self, | ||
| base_url: str, | ||
| session: Optional[ClientSession] = None, | ||
| protocol: Protocol = Protocol.MCP, | ||
| ): | ||
| self._mcp_base_url = f"{base_url}/mcp/" | ||
| self._protocol_version = protocol.value | ||
| self._server_version: Optional[str] = None | ||
|
|
||
| self._manage_session = session is None | ||
| self._session = session or ClientSession() | ||
| self._init_lock = asyncio.Lock() | ||
| self._init_task: Optional[asyncio.Task] = None | ||
|
|
||
| async def _ensure_initialized(self): | ||
| """Ensures the session is initialized before making requests.""" | ||
| async with self._init_lock: | ||
| if self._init_task is None: | ||
| self._init_task = asyncio.create_task(self._initialize_session()) | ||
| await self._init_task | ||
|
|
||
| @property | ||
| def base_url(self) -> str: | ||
| return self._mcp_base_url | ||
|
|
||
| def _convert_tool_schema(self, tool_data: dict) -> ToolSchema: | ||
| """Converts a raw MCP tool dictionary into the Toolbox ToolSchema.""" | ||
| parameters = [] | ||
| input_schema = tool_data.get("inputSchema", {}) | ||
| properties = input_schema.get("properties", {}) | ||
| required = input_schema.get("required", []) | ||
|
|
||
| for name, schema in properties.items(): | ||
| additional_props = schema.get("additionalProperties") | ||
| if isinstance(additional_props, dict): | ||
| additional_props = AdditionalPropertiesSchema( | ||
| type=additional_props["type"] | ||
| ) | ||
| else: | ||
| additional_props = True | ||
| parameters.append( | ||
| ParameterSchema( | ||
| name=name, | ||
| type=schema["type"], | ||
| description=schema.get("description", ""), | ||
| required=name in required, | ||
| additionalProperties=additional_props, | ||
| ) | ||
| ) | ||
|
|
||
| return ToolSchema( | ||
| description=tool_data.get("description") or "", parameters=parameters | ||
| ) | ||
|
|
||
| async def close(self): | ||
| async with self._init_lock: | ||
| if self._init_task: | ||
| try: | ||
| await self._init_task | ||
| except Exception: | ||
| # If initialization failed, we can still try to close. | ||
| pass | ||
| if self._manage_session and self._session and not self._session.closed: | ||
| await self._session.close() | ||
|
|
||
| @abstractmethod | ||
| async def _initialize_session(self): | ||
| """Initializes the MCP session.""" | ||
| pass |
168 changes: 168 additions & 0 deletions
168
packages/toolbox-core/src/toolbox_core/mcp_transport/v20241105/mcp.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,168 @@ | ||
| # Copyright 2025 Google LLC | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| from typing import Mapping, Optional, TypeVar | ||
|
|
||
| from pydantic import BaseModel | ||
|
|
||
| from ... import version | ||
| from ...protocol import ManifestSchema | ||
| from ..transport_base import _McpHttpTransportBase | ||
| from . import types | ||
|
|
||
| ReceiveResultT = TypeVar("ReceiveResultT", bound=BaseModel) | ||
|
|
||
|
|
||
| class McpHttpTransportV20241105(_McpHttpTransportBase): | ||
| """Transport for the MCP v2024-11-05 protocol.""" | ||
|
|
||
| async def _send_request( | ||
| self, | ||
| url: str, | ||
| request: types.MCPRequest[ReceiveResultT] | types.MCPNotification, | ||
| headers: Optional[Mapping[str, str]] = None, | ||
| ) -> ReceiveResultT | None: | ||
| """Sends a JSON-RPC request to the MCP server.""" | ||
| params = ( | ||
| request.params.model_dump(mode="json", exclude_none=True) | ||
| if isinstance(request.params, BaseModel) | ||
| else request.params | ||
| ) | ||
| rpc_msg: BaseModel | ||
| if isinstance(request, types.MCPNotification): | ||
| rpc_msg = types.JSONRPCNotification(method=request.method, params=params) | ||
| else: | ||
| rpc_msg = types.JSONRPCRequest(method=request.method, params=params) | ||
|
|
||
| payload = rpc_msg.model_dump(mode="json", exclude_none=True) | ||
|
|
||
| async with self._session.post( | ||
| url, json=payload, headers=dict(headers or {}) | ||
| ) as response: | ||
| if not response.ok: | ||
| error_text = await response.text() | ||
| raise RuntimeError( | ||
| f"API request failed with status {response.status} " | ||
| f"({response.reason}). Server response: {error_text}" | ||
| ) | ||
|
|
||
| if response.status == 204 or response.content.at_eof(): | ||
| return None | ||
|
|
||
| json_resp = await response.json() | ||
|
|
||
| # Check for JSON-RPC Error | ||
| if "error" in json_resp: | ||
| try: | ||
| err = types.JSONRPCError.model_validate(json_resp).error | ||
| raise RuntimeError( | ||
| f"MCP request failed with code {err.code}: {err.message}" | ||
| ) | ||
| except Exception: | ||
| raise RuntimeError(f"MCP request failed: {json_resp.get('error')}") | ||
|
|
||
| # Parse Result | ||
| if isinstance(request, types.MCPRequest): | ||
| try: | ||
| rpc_resp = types.JSONRPCResponse.model_validate(json_resp) | ||
| return request.get_result_model().model_validate(rpc_resp.result) | ||
| except Exception as e: | ||
| raise RuntimeError(f"Failed to parse JSON-RPC response: {e}") | ||
| return None | ||
|
|
||
| async def _initialize_session(self): | ||
| """Initializes the MCP session.""" | ||
| params = types.InitializeRequestParams( | ||
| protocolVersion=self._protocol_version, | ||
| capabilities=types.ClientCapabilities(), | ||
| clientInfo=types.Implementation( | ||
| name="toolbox-python-sdk", version=version.__version__ | ||
| ), | ||
| ) | ||
|
|
||
| result = await self._send_request( | ||
| url=self._mcp_base_url, request=types.InitializeRequest(params=params) | ||
| ) | ||
|
|
||
| self._server_version = result.serverInfo.version | ||
| if result.protocolVersion != self._protocol_version: | ||
| raise RuntimeError( | ||
| f"MCP version mismatch: client does not support server version {result.protocolVersion}" | ||
| ) | ||
| if not result.capabilities.tools: | ||
| if self._manage_session: | ||
| await self.close() | ||
| raise RuntimeError("Server does not support the 'tools' capability.") | ||
|
|
||
| await self._send_request( | ||
| url=self._mcp_base_url, request=types.InitializedNotification() | ||
| ) | ||
|
|
||
| async def tools_list( | ||
| self, | ||
| toolset_name: Optional[str] = None, | ||
| headers: Optional[Mapping[str, str]] = None, | ||
| ) -> ManifestSchema: | ||
| """Lists available tools from the server using the MCP protocol.""" | ||
| await self._ensure_initialized() | ||
|
|
||
| url = self._mcp_base_url + (toolset_name if toolset_name else "") | ||
| result = await self._send_request( | ||
| url=url, request=types.ListToolsRequest(), headers=headers | ||
| ) | ||
| if result is None: | ||
| raise RuntimeError("Failed to list tools: No response from server.") | ||
|
|
||
| tools_map = { | ||
| t.name: self._convert_tool_schema(t.model_dump(mode="json", by_alias=True)) | ||
| for t in result.tools | ||
| } | ||
| if self._server_version is None: | ||
| raise RuntimeError("Server version not available.") | ||
|
|
||
| return ManifestSchema(serverVersion=self._server_version, tools=tools_map) | ||
|
|
||
| async def tool_get( | ||
| self, tool_name: str, headers: Optional[Mapping[str, str]] = None | ||
| ) -> ManifestSchema: | ||
| """Gets a single tool from the server by listing all and filtering.""" | ||
| manifest = await self.tools_list(headers=headers) | ||
|
|
||
| if tool_name not in manifest.tools: | ||
| raise ValueError(f"Tool '{tool_name}' not found.") | ||
|
|
||
| return ManifestSchema( | ||
| serverVersion=manifest.serverVersion, | ||
| tools={tool_name: manifest.tools[tool_name]}, | ||
| ) | ||
|
|
||
| async def tool_invoke( | ||
| self, tool_name: str, arguments: dict, headers: Optional[Mapping[str, str]] | ||
| ) -> str: | ||
| """Invokes a specific tool on the server using the MCP protocol.""" | ||
| await self._ensure_initialized() | ||
|
|
||
| result = await self._send_request( | ||
| url=self._mcp_base_url, | ||
| request=types.CallToolRequest( | ||
| params=types.CallToolRequestParams(name=tool_name, arguments=arguments) | ||
| ), | ||
| headers=headers, | ||
| ) | ||
| if result is None: | ||
| raise RuntimeError( | ||
| f"Failed to invoke tool '{tool_name}': No response from server." | ||
| ) | ||
|
|
||
| return "".join(c.text for c in result.content if c.type == "text") or "null" |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.