✨ feat!(llm): 重构并升级大语言模型服务为全新 AI 智能体框架 (#2146)

* ✨ feat!(llm): 重构并升级大语言模型服务为全新 AI 智能体框架

- 【重构】将原 services/llm 重构并迁移至全新的 services/ai 架构,提供向下兼容垫片
- 【新增】引入 Agent、Team、Workflow 三大智能体与工作流编排范式
- 【新增】引入基于 RAG 的长期向量记忆与中期槽位记忆系统
- 【新增】引入基于 Docker 的安全代码执行沙箱环境
- 【新增】支持 MCP 协议,允许动态管理和调用 MCP 服务
- 【新增】引入输入输出安全合规护栏与自愈反思机制
- 【优化】重构并优化多厂商 API 适配器 (Gemini, OpenAI, DeepSeek, GLM 等)
- 【优化】优化日志脱敏与 Token 预估机制
- 【移除】移除旧版 llm default 和 llm reset-key 命令,新增 llm mcp 管理命令

* 🔧 chore(deps): 更新项目依赖与配置

- 添加 mcp、jieba 和 aiodocker 依赖到配置文件及 requirements.txt
- 在 pyright 配置中设置 reportMissingImports 为 none
- 调整 .gitignore 中 resources 目录的忽略规则

* ♻️ refactor(tools): 重构工具终止机制并清理知识库日志输出

- 统一使用 `context.state["__end_run__"]` 替代 `EndRunResult` 控制任务结束
- 移除文件系统和向量知识库检索工具中 `ToolResult` 的 `.with_log` 调用
- 调整指令处理器(Directive)的返回值为 `tool_res.output`
- 修复部分类型检查警告并优化联合类型判断语法

* ♻️ refactor(tools): 重构工具副作用指令与控制流熔断机制

- 引入 `DirectivePayload` 及 `ToolResult` 的子类以结构化表达工具副作用
- 移除通过 `context.state` 传递魔术变量的隐式控制流设计
- 重构 `DirectiveManager` 处理器接口,直接在处理器中修改 `AgentState` 并构建 `AgentRunResult`
- 在 `StandardAgentExecutor` 中统一通过 `directive_manager` 调度工具返回的副作用指令
- 补全 `MessageBuilder` 中部分核心方法的文档注释

* 🐛 fix(sandbox): 修复 Docker 沙箱容器状态检测与会话清理逻辑

-【修复】修正 `is_alive` 中直接读取私有属性的问题,改用 `show()` 返回值
-【修复】解决 `execute_code` 中缓存的执行器与当前会话不一致的问题
-【优化】在清理工作区前增加容器存活检测,避免向已死容器发送请求
-【优化】创建容器时增加运行状态校验,若已停止则自动从缓存中移除并重建
-【优化】优化容器销毁和清理逻辑,静默处理容器不存在 (404) 的异常

* 📝 docs(core): 补充核心模块初始化方法的文档注释

* 🚨 auto fix by pre-commit hooks

---------

Co-authored-by: webjoin111 <455457521@qq.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
Rumio
2026-07-03 08:53:56 +08:00
committed by GitHub
co-authored by webjoin111 pre-commit-ci[bot]
parent bdc1374848
commit 80fc5b86a7
223 changed files with 41885 additions and 10006 deletions
@@ -0,0 +1,15 @@
from .blackboard import BlackboardToolkit
from .hitl import HITLToolkit
from .memory import MemoryManagementToolkit
from .rest_api import RestApiToolkit
from .sandbox import SandboxToolkit
from .slots import MemorySlotToolkit
__all__ = [
"BlackboardToolkit",
"HITLToolkit",
"MemoryManagementToolkit",
"MemorySlotToolkit",
"RestApiToolkit",
"SandboxToolkit",
]
@@ -0,0 +1,72 @@
from typing import Any
from pydantic import Field, create_model
from zhenxun.services.ai.core.exceptions import ToolRetryError
from zhenxun.services.ai.run.blackboard import BlackboardManager
from zhenxun.services.ai.tools.core.tool import FunctionTool
from zhenxun.services.ai.tools.core.toolkit import BaseToolkit
from zhenxun.services.ai.tools.models import ToolOptions, ToolResult
from zhenxun.utils.pydantic_compat import model_fields
class BlackboardToolkit(BaseToolkit):
"""
黑板工具箱,将底层的 BlackboardManager 桥接给大模型,提供动态的读写工具。
"""
default_prefix = ""
def __init__(self, manager: BlackboardManager, **kwargs: Any):
super().__init__(**kwargs)
self.manager = manager
self._injected_tools.extend(self._create_dynamic_tools())
def _create_dynamic_tools(self) -> list[FunctionTool]:
async def read_blackboard() -> ToolResult:
"""读取当前团队的全局共享黑板状态。"""
content = await self.manager.read()
return ToolResult(output=content)
read_tool = FunctionTool(
func=read_blackboard,
name="read_blackboard",
description="读取当前团队的全局共享黑板最新状态。当你需要获取最新数据或了解现状时请调用此工具。",
)
optional_fields = {}
for field in model_fields(self.manager.schema):
desc = (
getattr(field.field_info, "description", None)
if hasattr(field, "field_info")
else None
)
optional_fields[field.name] = (
field.annotation | None,
Field(default=None, description=desc),
)
UpdateSchema = create_model(
f"Update_{self.manager.schema.__name__}", **optional_fields
)
async def update_blackboard(**kwargs) -> ToolResult:
valid_kwargs = {k: v for k, v in kwargs.items() if v is not None}
if not valid_kwargs:
return ToolResult(
output="警告:没有提供任何有效的更新字段,黑板状态未发生改变。"
)
try:
res = await self.manager.update(**valid_kwargs)
return ToolResult(output=res)
except ValueError as e:
raise ToolRetryError(str(e))
update_tool = FunctionTool(
func=update_blackboard,
name="update_blackboard",
description="局部更新全局黑板状态。请仅提供你需要修改的字段即可,不需要改变的字段请留空。",
settings=ToolOptions(args_schema=UpdateSchema),
)
return [read_tool, update_tool]
@@ -0,0 +1,56 @@
from zhenxun.services.ai.core.exceptions import AbortException, ToolFatalError
from zhenxun.services.ai.core.stream_events import ToolStreamChunkEvent
from zhenxun.services.ai.run import Inject, RunContext
from zhenxun.services.ai.tools.core.decorators import tool
from zhenxun.services.ai.tools.core.toolkit import BaseToolkit
from zhenxun.services.ai.tools.models import ToolResult
from zhenxun.services.log import logger
class HITLToolkit(BaseToolkit):
"""
Human-in-the-Loop (HITL) 工具箱。
提供大模型主动挂起执行流并向用户提问求助的能力。
"""
default_prefix = ""
default_instructions = (
"## 人机协同求助系统\n"
"你拥有向用户发起提问的权限。当你遇到困难时(例如缺少关键信息、连续执行某个操作出错、找不到文件等),"
"可以使用 `ask_user_for_help` 工具请求用户协助。"
"用户回答后,你将收到答案并可以继续任务。"
)
@tool(
name="ask_user_for_help",
description="向当前对话的用户提出问题以获取信息或指导。当你无法独立完成任务时,请调用此工具。",
)
async def ask_user_for_help(
self,
question: str,
hitl: Inject.HITL,
context: RunContext,
) -> ToolResult:
try:
user_reply = await hitl.ask_text(
f"🤖 [AI 提问]\n{question}\n\n(请在 60 秒内回复,或回复'取消')",
timeout=60.0,
)
except ToolFatalError:
return ToolResult(
output="用户未在规定时间内回复 (超时未响应),"
"你必须尝试自行解决或中止任务。",
).as_error()
except AbortException as e:
raise e
logger.info(f"收到用户求助回复: {user_reply}")
if context and context.run.event_bus:
await context.run.event_bus.emit(
ToolStreamChunkEvent(
tool_name=context.call.tool_name, content="🗣️ 已收到用户的回复"
)
)
return ToolResult(output=f"用户提供的回复是: {user_reply}")
@@ -0,0 +1,324 @@
from typing import Any, Literal, Optional
from pydantic import Field, create_model
from zhenxun.services.ai.context.memory.manager import memory_manager
from zhenxun.services.ai.context.memory.models import MemoryConfig
from zhenxun.services.ai.context.memory.types import SessionMetadata
from zhenxun.services.ai.run.context import RunContext
from zhenxun.services.ai.tools.core.decorators import tool
from zhenxun.services.ai.tools.core.tool import BaseTool
from zhenxun.services.ai.tools.core.toolkit import BaseToolkit
from zhenxun.services.ai.tools.models import ToolOptions, ToolResult
from zhenxun.services.log import logger
class MemoryManagementToolkit(BaseToolkit):
"""
主动记忆管理工具箱 (Agentic Memory Toolkit)。
赋予大模型自主存取、修改和删除用户长期设定的能力。
"""
default_prefix = ""
class Config:
"""声明式配置:该工具箱下的所有工具默认静默"""
shared_options = ToolOptions(silent=True)
default_instructions = """\
## 🧠 长期记忆管理系统 (Long-Term Memory)
该系统是你的「无限档案馆」。⚠️ **注意:系统默认不会主动向你提供所有历史信息,你必须通过主动搜索来回忆。**
### 📝 何时使用长期记忆?
- **记录离散事实与经验**:当需要记录某个独立事件、历史经验、问题解决方案或具体事实时(使用 `save_memory`)。
- **寻找历史线索**:当遇到未知情况,或用户提及过去的事情、特定设定时,必须主动检索历史库(使用 `search_memory`)。
### ⚙️ 操作规范
1. **隐式记录**:当接收到值得记忆的重要信息时,请静默记录。除非用户主动提问,否则无需向用户显式汇报"我已记住"。
2. **按需更新**:如果发现某项历史记录已过时或状态发生扭转,请先检索出它的 ID,再进行修改(使用 `update_memory`)或废弃(使用 `delete_memory`)。
3. **精准提炼**:保存记忆时请提炼核心价值,避免保存无意义的闲聊。\
""" # noqa: E501
def __init__(
self,
memory_config: MemoryConfig | None = None,
namespace: str | None = None,
**kwargs: Any,
):
"""
初始化主动记忆管理工具箱。
参数:
memory_config: 记忆系统的全局配置对象,为空则使用全局默认。
namespace: 当前隔离环境的命名空间。
kwargs: 其他透传给 BaseToolkit 的参数。
"""
super().__init__(**kwargs)
self.memory_config = memory_config
self._namespace = namespace
def _get_runtime_meta_and_scope(
self, context: RunContext, scope_name: str | None = None
) -> tuple[Any, SessionMetadata]:
"""动态获取当前运行时的数据库实例与会话元信息,实现无状态化"""
ns = self._namespace or getattr(context.session, "namespace", "global")
scope = memory_manager.get_long_term_memory(self.memory_config, namespace=ns)
scope_builder = None
if (
self.memory_config
and self.memory_config.long_term
and self.memory_config.long_term.scopes
):
scopes_dict = self.memory_config.long_term.scopes
if not scope_name:
scope_builder = next(iter(scopes_dict.values()))
else:
scope_builder = scopes_dict.get(scope_name)
if not scope_builder:
scope_builder = getattr(self.memory_config, "base_isolation", None)
if not scope_builder:
from zhenxun.services.ai.context.memory.types import Isolation
scope_builder = Isolation.AGENT_USER()
selector = scope_builder.resolve(
deps=context.deps,
prefix="",
default_namespace=ns,
default_agent=context.run.agent_name,
)
parts = selector.get_scope_parts()
all_scopes = {"/"}
current_path = ""
for part in parts:
current_path += f"/{part}"
all_scopes.add(current_path)
accessible_scopes = list(all_scopes)
accessible_scopes.sort(key=lambda x: len(x.split("/")))
scope_name_mapping = {}
if (
self.memory_config
and self.memory_config.long_term
and self.memory_config.long_term.scopes
):
for name, builder in self.memory_config.long_term.scopes.items():
sel = builder.resolve(
deps=context.deps,
prefix="",
default_namespace=ns,
default_agent=context.run.agent_name,
)
scope_name_mapping[sel.scope_prefix] = name
session_meta = SessionMetadata(
session_id=context.session_id or "default_session",
selector=selector,
scope_prefix=selector.scope_prefix,
accessible_scopes=accessible_scopes,
scope_name_mapping=scope_name_mapping,
)
return scope, session_meta
async def get_tools(self, context: RunContext | None = None) -> dict[str, BaseTool]:
tools = await super().get_tools(context)
if not self.memory_config or not self.memory_config.long_term.enable:
return tools
scopes_dict = self.memory_config.long_term.scopes
if not scopes_dict:
return tools
scope_keys = tuple(scopes_dict.keys())
if len(scope_keys) > 1:
ScopeType = Literal[scope_keys]
SaveArgs = create_model(
"SaveMemoryArgs",
content=(str, Field(..., description="要保存的记忆内容")),
importance=(float, Field(default=0.5, description="重要性(0-1)")),
scope=(
ScopeType,
Field(..., description="选择记忆存储的隔离分区"),
),
)
SearchArgs = create_model(
"SearchMemoryArgs",
query=(str, Field(..., description="搜索关键词或问题")),
filters=(
str | None,
Field(default=None, description="可选的元数据过滤JSON字符串"),
),
scope=(
Optional[ScopeType], # noqa
Field(
default=None,
description="搜索特定的分区。留空则跨所有有权分区混合检索!",
),
),
)
UpdateArgs = create_model(
"UpdateMemoryArgs",
record_id=(str, Field(..., description="要更新的记忆ID")),
new_content=(str, Field(..., description="新的记忆内容")),
importance=(float, Field(default=0.5, description="重要性(0-1)")),
scope=(
ScopeType,
Field(..., description="指定该记忆所在的隔离分区"),
),
)
DeleteArgs = create_model(
"DeleteMemoryArgs",
record_id=(str, Field(..., description="要删除的记忆ID")),
scope=(
ScopeType,
Field(..., description="指定该记忆所在的隔离分区"),
),
)
for t_name, t in tools.items():
if t_name.endswith("save_memory"):
t.args_schema = SaveArgs
elif t_name.endswith("search_memory"):
t.args_schema = SearchArgs
elif t_name.endswith("update_memory"):
t.args_schema = UpdateArgs
elif t_name.endswith("delete_memory"):
t.args_schema = DeleteArgs
return tools
@tool(
name="save_memory",
description="保存关于用户的重要设定、偏好或事实到长期记忆中。",
)
async def save_memory(
self, content: str, context: RunContext, importance: float = 0.5, **kwargs
) -> ToolResult:
scope_name = kwargs.get("scope")
scope, meta = self._get_runtime_meta_and_scope(context, scope_name)
if not scope:
return ToolResult(output="错误:未启用或未配置长期记忆后端。").as_error()
await scope.remember(session=meta, content=content, importance=importance)
logger.debug(
f"[Agentic Memory] AI 主动存入记忆: {content} (重要性: {importance})"
)
return ToolResult(output="记忆已成功排入系统后台合并存入。")
@tool(
name="search_memory",
description=(
"在长期记忆中主动检索关于用户的设定和事实。"
"支持自然语言模糊搜索。"
"可选的 filters 参数用于元数据精确匹配。如果不需要过滤,"
"请直接省略该参数,不要传入空字典或空字符串。"
),
)
async def search_memory(
self,
query: str,
context: RunContext,
**kwargs,
) -> ToolResult:
scope_name = kwargs.get("scope")
scope, meta = self._get_runtime_meta_and_scope(context, scope_name)
if not scope:
return ToolResult(output="错误:未启用或未配置长期记忆后端。").as_error()
filters = kwargs.get("filters")
parsed_filters = None
if isinstance(filters, dict):
parsed_filters = filters
elif isinstance(filters, str) and filters.strip() and filters.strip() != "{}":
import json
try:
parsed_filters = json.loads(filters)
except Exception:
pass
if scope_name is not None:
meta.accessible_scopes = [meta.scope_prefix]
matches = await scope.recall(
session=meta,
query=query,
limit=5,
metadata_filter=parsed_filters,
)
if not matches:
return ToolResult(
output="未检索到相关记忆。这可能是你们关于此话题的首次探讨,请直接根据常识回答或向用户确认。"
)
results = [
(
f"ID: {m.record.id} | 内容: {m.record.content} | "
f"重要性: {m.record.metadata.get('importance', 0.5)}"
)
for m in matches
]
return ToolResult(output="检索到的记忆如下:\n" + "\n".join(results))
@tool(
name="update_memory",
description="更新指定ID的长期记忆内容。当你发现用户的某个旧设定发生改变时,请使用此工具覆盖旧记忆。",
)
async def update_memory(
self,
record_id: str,
new_content: str,
context: RunContext,
importance: float = 0.5,
**kwargs,
) -> ToolResult:
scope_name = kwargs.get("scope")
scope, meta = self._get_runtime_meta_and_scope(context, scope_name)
if not scope:
return ToolResult(output="错误:未启用或未配置长期记忆后端。").as_error()
success = await scope.update(
session=meta,
record_id=record_id,
new_content=new_content,
importance=importance,
)
if success:
logger.debug(
f"[Agentic Memory] AI 主动更新记忆: {record_id} -> {new_content}"
)
return ToolResult(
output=f"记忆 {record_id} 更新成功。新内容:{new_content}"
)
return ToolResult(
output=f"更新失败:未找到ID为 {record_id} 的记忆。"
).as_error()
@tool(
name="delete_memory",
description="根据记忆的唯一ID删除已经作废或过期的用户记忆。",
)
async def delete_memory(
self, record_id: str, context: RunContext, **kwargs
) -> ToolResult:
scope_name = kwargs.get("scope")
scope, meta = self._get_runtime_meta_and_scope(context, scope_name)
if not scope:
return ToolResult(output="错误:未启用或未配置长期记忆后端。").as_error()
deleted_count = await scope.forget(session=meta, record_ids=[record_id])
if deleted_count > 0:
logger.info(f"[Agentic Memory] AI 主动删除记忆: {record_id}")
return ToolResult(output=f"记忆 {record_id} 删除成功。")
return ToolResult(
output=f"删除失败:未找到ID为 {record_id} 的记忆。"
).as_error()
@@ -0,0 +1,129 @@
from zhenxun.services.ai.tools.core.tool import ServerSideTool
class WebSearchTool(ServerSideTool):
"""原生网页搜索工具"""
type_id = "web_search"
def __init__(
self,
name: str = "google_search",
description: str = "使用 Google 搜索查找实时信息。",
dynamic_threshold: float | None = None,
domain_filters: list[str] | None = None,
):
super().__init__(name=name, description=description)
self.dynamic_threshold = dynamic_threshold
self.domain_filters = domain_filters
class CodeExecutionTool(ServerSideTool):
"""原生代码沙箱执行工具"""
type_id = "code_execution"
def __init__(self, name: str = "code_execution", timeout: int | None = None):
super().__init__(
name=name, description="执行 Python 代码来解决数学或数据问题。"
)
self.timeout = timeout
class ComputerUseTool(ServerSideTool):
"""原生的桌面环境控制工具"""
type_id = "computer_use"
def __init__(self, display_width_px: int = 1024, display_height_px: int = 768):
super().__init__(name="computer_use", description="控制电脑用户界面。")
self.display_width_px = display_width_px
self.display_height_px = display_height_px
class FileSearchTool(ServerSideTool):
"""原生文件检索工具"""
type_id = "file_search"
def __init__(self, name: str = "file_search"):
super().__init__(name=name, description="搜索已索引的文件。")
class GoogleMapsTool(ServerSideTool):
"""原生谷歌地图检索工具"""
type_id = "google_map"
def __init__(self):
super().__init__(name="google_map", description="搜索 Google 地图数据。")
class UrlContextTool(ServerSideTool):
"""原生 URL 上下文拉取工具"""
type_id = "url_context"
def __init__(self):
super().__init__(name="url_context", description="从指定 URL 获取上下文信息。")
class Native:
"""
云端原生工具命名空间工厂 (Namespace Factory)。
为开发者提供统一的云端内置工具调用入口,享受顶级 IDE 补全体验。
此类工具仅会向大模型提供描述,物理执行发生在各大模型厂商的服务端。
"""
@classmethod
def web_search(
cls,
name: str = "google_search",
description: str = "使用 Google 搜索查找实时信息。",
dynamic_threshold: float | None = None,
domain_filters: list[str] | None = None,
) -> WebSearchTool:
"""
原生网页搜索引擎工具 (如 Google Search, Bing)。
"""
return WebSearchTool(name, description, dynamic_threshold, domain_filters)
@classmethod
def code_execution(
cls, name: str = "code_execution", timeout: int | None = None
) -> CodeExecutionTool:
"""
原生代码沙箱执行工具 (如 Gemini Code Execution, OpenAI Advanced Data Analysis)。
"""
return CodeExecutionTool(name, timeout)
@classmethod
def computer_use(
cls, display_width_px: int = 1024, display_height_px: int = 768
) -> ComputerUseTool:
"""
原生桌面环境控制工具 (如 Claude Computer Use)。
"""
return ComputerUseTool(display_width_px, display_height_px)
@classmethod
def file_search(cls, name: str = "file_search") -> FileSearchTool:
"""
原生文件检索工具。
"""
return FileSearchTool(name)
@classmethod
def google_map(cls) -> GoogleMapsTool:
"""
原生谷歌地图检索工具 (Gemini 独占)。
"""
return GoogleMapsTool()
@classmethod
def url_context(cls) -> UrlContextTool:
"""
原生 URL 上下文拉取工具。
"""
return UrlContextTool()
@@ -0,0 +1,132 @@
import json
from typing import Any, Literal
import httpx
from zhenxun.services.ai.core.stream_events import ToolStreamChunkEvent
from zhenxun.services.ai.run import RunContext
from zhenxun.services.ai.tools.core.decorators import tool
from zhenxun.services.ai.tools.core.toolkit import BaseToolkit
from zhenxun.services.ai.tools.models import ToolResult
from zhenxun.services.log import logger
class RestApiToolkit(BaseToolkit):
"""
通用 REST API 工具箱。
让大模型无需编写任何代码即可调用外部 HTTP 接口获取或提交数据。
"""
default_prefix = ""
default_instructions = (
"你拥有调用外部 REST API 接口的能力。\n"
"1. 使用 `make_request` 工具发起 HTTP 请求。\n"
"2. 如果请求失败,请分析返回的错误信息。可能需要调整请求参数 (params) 或请求体 (json_data)。\n" # noqa: E501
"3. 务必根据返回的 JSON 数据,提取对用户有用的信息进行回答。"
)
def __init__(
self,
base_url: str = "",
default_headers: dict[str, str] | None = None,
default_params: dict[str, Any] | None = None,
timeout: int = 30,
**kwargs: Any,
):
"""
初始化通用 REST API 工具箱。
参数:
base_url: 接口基础 URL,默认发起请求时将与其拼接。
default_headers: 发起请求时携带的默认 Headers 请求头。
default_params: 发起请求时携带的默认 URL 查询参数。
timeout: 请求超时时间,单位为秒。
kwargs: 其他透传给 BaseToolkit 的参数。
"""
super().__init__(**kwargs)
self.base_url = base_url.rstrip("/") if base_url else ""
self.default_headers = default_headers or {}
self.default_params = default_params or {}
self.timeout = timeout
@tool(name="make_request")
async def make_request(
self,
endpoint: str,
method: Literal["GET", "POST", "PUT", "DELETE", "PATCH"] = "GET",
params: dict[str, Any] | None = None,
json_data: dict[str, Any] | None = None,
context: RunContext | None = None,
) -> ToolResult:
"""
发起 HTTP 请求获取外部数据。
Args:
endpoint: API的路径(若初始化了base_url,将自动拼接)。如果是一个完整的URL(以http开头),则直接请求该URL。
method: HTTP 请求方法。
params: URL 查询参数字典。
json_data: JSON 请求体字典。
""" # noqa: E501
if endpoint.startswith("http://") or endpoint.startswith("https://"):
url = endpoint
else:
url = (
f"{self.base_url}/{endpoint.lstrip('/')}" if self.base_url else endpoint
)
merged_params = self.default_params.copy()
if params:
merged_params.update(params)
logger.info(f"🌐 [RestApiToolkit] 正在请求 {method} {url}")
try:
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.request(
method=method,
url=url,
params=merged_params,
json=json_data,
headers=self.default_headers,
)
try:
response_data = response.json()
except json.JSONDecodeError:
response_data = response.text
result_dict = {"status_code": response.status_code, "data": response_data}
is_error = response.status_code >= 400
result = ToolResult(output=result_dict)
if context and context.run.event_bus:
msg = (
f"🌐 已调用 API: {url}"
if not is_error
else f"❌ API 调用失败 (Status: {response.status_code})"
)
await context.run.event_bus.emit(
ToolStreamChunkEvent(
tool_name=context.call.tool_name if context else "make_request",
content=msg,
)
)
if is_error:
result = result.as_error()
return result
except Exception as e:
logger.error(f"RestApiToolkit 请求失败: {e}")
if context and context.run.event_bus:
await context.run.event_bus.emit(
ToolStreamChunkEvent(
tool_name=context.call.tool_name if context else "make_request",
content="❌ API 网络请求发生框架级错误",
)
)
return ToolResult(
output=f"网络请求失败: {type(e).__name__} - {e}"
).as_error()
@@ -0,0 +1,425 @@
import asyncio
from typing import Any, Protocol
from zhenxun.services.ai.core.stream_events import ToolStreamChunkEvent, UserCustomEvent
from zhenxun.services.ai.run import Inject, RunContext
from zhenxun.services.ai.sandbox.models import (
SandboxBlueprint,
SandboxExecutionResult,
)
from zhenxun.services.ai.tools.core.decorators import silent, tool
from zhenxun.services.ai.tools.core.toolkit import BaseToolkit
from zhenxun.services.ai.tools.models import ToolResult
from zhenxun.services.log import logger
from zhenxun.utils.pydantic_compat import model_copy
class PythonPluginProtocol(Protocol):
@property
def supports_state(self) -> bool: ...
async def execute(
self,
code: str,
timeout: int = 30,
injected_code: str | None = None,
) -> SandboxExecutionResult: ...
class SandboxToolkit(BaseToolkit):
default_prefix = ""
default_instructions = (
"## 🖥️ 沙箱工作区交互规范\n"
"你拥有物理隔离的沙箱环境。请严格遵循以下调度规则:\n"
"1. **短时/非交互任务**:直接使用 `execute_code`(如数据计算、算法运行)。"
"⚠️ 该工具**严禁包含 `input()`** 等阻塞式交互。\n"
"2. **长驻/交互式任务**:若需运行 Web Server 或含 `input()` 的交互程序,"
"**必须**:\n"
" - 先用 `write_sandbox_file` 将代码保存至当前工作区。\n"
" - 再用 `execute_terminal_command(is_interactive=True)` 启动并挂起进程。\n"
" - 通过 `send_sandbox_input` / `read_sandbox_screen` 与屏幕画面交互。\n"
"3. **终端互斥锁**:虚拟终端只能单线程运行前台程序。"
"若程序报错或死循环卡死,**必须**先调用 `interrupt_sandbox` "
"打断它,方可进行后续修改。"
)
def __init__(
self,
blueprint: SandboxBlueprint | None = None,
sandbox_session_id: str | None = None,
**kwargs: Any,
):
"""
初始化沙箱工作区工具箱。
参数:
blueprint: 沙箱蓝图配置对象,包含沙箱运行时的各种参数定义(例如是否需要状态、端口映射等)。
sandbox_session_id: 显式指定的沙箱会话 ID。如果不指定,则自动与当前执行上下文绑定。
kwargs: 其他透传给 BaseToolkit 的参数。
""" # noqa: E501
super().__init__(**kwargs)
self.blueprint = blueprint or SandboxBlueprint()
self.sandbox_session_id = sandbox_session_id
async def _get_session(self, context: RunContext, sandbox: Any) -> Any:
"""获取或创建沙箱会话实例"""
session_id = (
self.sandbox_session_id or context.session_id or "default_sandbox_session"
)
return await sandbox.get_or_create_session(session_id, self.blueprint)
def _get_pty(self, context: RunContext) -> Any:
"""从上下文获取虚拟终端"""
session_id = self.sandbox_session_id or context.session_id or "default"
return context.session.shared_state.get(f"pty_{session_id}")
@tool(
name="execute_code",
description=(
"在沙箱环境中执行代码。\n"
"支持多语言执行,请在 language 参数中指定具体的编程语言"
"(如 python, bash 等)。\n"
"默认超时时间为 45 秒。如果你的代码需要更长的时间或等待用户输入,"
"它将被挂在后台运行。\n"
"你将收到目前的屏幕输出,并可以决定后续操作。"
),
)
async def execute_code(
self,
code: str,
language: str,
context: RunContext,
sandbox: Inject.Sandbox,
) -> ToolResult:
session_id = (
self.sandbox_session_id or context.session_id or "default_sandbox_session"
)
logger.info(
f"大模型请求执行 {language} 代码 (Session: {session_id}, "
f"长度: {len(code)} 字符)"
)
from zhenxun.services.ai.sandbox.runtimes import CodeExecutorRegistry
ns = getattr(context.session, "namespace", "global") if context else "global"
supported = CodeExecutorRegistry.get_supported_languages(ns)
if CodeExecutorRegistry._normalize_lang(language) not in supported:
from zhenxun.services.ai.core.exceptions import ToolRetryError
raise ToolRetryError(
f"当前沙箱不支持该语言 '{language}'。支持的语言有: {supported}。"
"请换用支持的语言重新编写代码!"
)
bp = model_copy(self.blueprint, deep=True)
if context.run.event_bus:
await context.run.event_bus.emit(
ToolStreamChunkEvent(
tool_name="Sandbox", content="正在分析代码依赖并分配沙箱环境..."
)
)
executor = await self._get_session(context, sandbox)
state_key = f"code_exec_{session_id}_{language}"
code_executor = context.session.shared_state.get(state_key)
if code_executor and getattr(code_executor, "session", None) is not executor:
code_executor = None
if not code_executor:
code_executor = CodeExecutorRegistry.create_executor(
language, bp.needs_state, executor, namespace=ns
)
context.session.shared_state[state_key] = code_executor
if context.run.event_bus:
await context.run.event_bus.emit(
ToolStreamChunkEvent(
tool_name="Sandbox",
content=f"沙箱已就绪,正在后台执行 {language} 代码...",
)
)
import re
clean_code = code.strip()
match = re.search(
r"^```[a-zA-Z0-9_-]*\r?\n(.*?)\r?\n```$",
clean_code,
re.IGNORECASE | re.DOTALL,
)
if match:
clean_code = match.group(1)
line_buffer = ""
async def _stream_output(stream_type: str, data: bytes):
nonlocal line_buffer
line_buffer += data.decode("utf-8", errors="replace")
try:
result = await code_executor.execute_code(
code=clean_code,
timeout=45,
on_output=_stream_output,
)
except Exception as e:
logger.error(f"沙箱执行框架异常: {e}")
from zhenxun.services.ai.core.exceptions import AbortException
raise AbortException(
reason=f"System Error: 容器执行环境不可用,异常信息: {e}",
display=f"❌ 沙箱框架异常: {e}",
)
output_parts = []
if result.exit_code != 0 or result.stderr:
output_parts.append(f"Exit Code: {result.exit_code}")
if result.stdout:
output_parts.append(f"Stdout:\n{result.stdout.strip()[:2000]}")
if result.stderr:
output_parts.append(f"Stderr:\n{result.stderr.strip()[:2000]}")
if result.error:
output_parts.append(f"System Error:\n{result.error}")
output_text = "\n".join(output_parts)
system_notice = ""
if result.is_timeout:
system_notice = (
"\n\n⚠️ 警告: 代码执行超时!进程仍在后台挂起,"
"若为死循环请立刻使用 `interrupt_sandbox`。"
)
if result.stderr and "StdinNotImplementedError" in result.stderr:
system_notice += (
"\n\n🚨 致命错误: 当前环境不支持 input()。"
"请将代码写入文件并通过 "
"execute_terminal_command(is_interactive=True) 运行!"
)
image_bytes_list = []
if getattr(result, "images", None):
import base64
for b64_str in result.images:
try:
image_bytes_list.append(base64.b64decode(b64_str))
except Exception:
pass
if getattr(result, "artifacts", None):
for filename, file_bytes in result.artifacts.items():
if filename.endswith((".png", ".jpg", ".jpeg")):
image_bytes_list.append(file_bytes)
from zhenxun.services.ai.core.messages import (
ImagePart,
LLMContentPart,
TextPart,
)
final_output: list[LLMContentPart] = [TextPart(text=output_text.strip())]
for img_bytes in image_bytes_list:
final_output.append(ImagePart(raw=img_bytes))
final_output_text = output_text.strip()
if system_notice:
final_output_text += f"\n\n{system_notice}"
final_output[0] = TextPart(text=final_output_text)
result = ToolResult(
output=final_output if len(final_output) > 1 else final_output_text
)
if len(image_bytes_list) > 0 and context and context.run.event_bus:
await context.run.event_bus.emit(UserCustomEvent(display=final_output))
return result
@tool(
name="execute_terminal_command",
description=(
"在沙箱的终端中执行 Shell 命令"
"(例如 `python3 script.py` 或 `npm start`)。\n"
"如果只是执行普通的短时非交互脚本,"
"保持 is_interactive=False 即可(执行速度极快且稳定)。\n"
"如果程序包含 `input()` 或需要长期驻留(如 Server),"
"请务必设置 is_interactive=True 开启虚拟屏幕模式!"
),
)
async def execute_terminal_command(
self,
command: str,
context: RunContext,
sandbox: Inject.Sandbox,
is_interactive: bool = False,
) -> ToolResult:
session_id = self.sandbox_session_id or context.session_id or "default"
if context.run.event_bus:
await context.run.event_bus.emit(
ToolStreamChunkEvent(
tool_name="Sandbox", content=f"正在虚拟终端执行命令: {command} ..."
)
)
executor = await self._get_session(context, sandbox)
if not is_interactive:
res = await executor.run_process(command)
if getattr(res, "is_timeout", False):
return ToolResult(
output=(
"⚠️ 警告: 命令执行超时!若程序需常驻或等待输入,"
f"请务必设置 is_interactive=True。\n"
f"Stdout: {res.stdout}\nStderr: {res.stderr}"
)
).as_error()
return ToolResult(
output=(
f"Exit Code: {res.exit_code}\n"
f"Stdout: {res.stdout}\n"
f"Stderr: {res.stderr}"
)
)
pty = self._get_pty(context)
if pty:
await pty.close()
interactive_session = await executor.create_pty_session()
context.session.shared_state[f"pty_{session_id}"] = interactive_session
try:
await interactive_session.start(command)
await asyncio.sleep(1.5)
screen = await interactive_session.read_output()
return ToolResult(
output=f"📺 虚拟终端已启动,屏幕快照:\n```text\n{screen}\n```"
)
except Exception as e:
return ToolResult(output=f"虚拟屏幕启动异常: {e}").as_error()
@tool(
name="send_sandbox_input",
description=(
"向当前沙箱中正在挂起运行的后台进程"
"(如等待 input() 的 Python 脚本)发送输入文本。\n"
"注意:你需要自己在文本末尾加上换行符 \\n 来模拟回车键。"
),
)
async def send_sandbox_input(self, text: str, context: RunContext) -> ToolResult:
interactive_session = self._get_pty(context)
if not interactive_session:
return ToolResult(
output=(
"错误:没有运行中的交互式虚拟屏幕。"
"请先调用 execute_terminal_command(is_interactive=True)。"
)
).as_error()
text = text.replace("\\n", "\n").replace("\\r", "\r").replace("\\t", "\t")
await interactive_session.send_input(text)
await asyncio.sleep(1.5)
output = await interactive_session.read_output(timeout=5)
if context and context.run.event_bus:
await context.run.event_bus.emit(
ToolStreamChunkEvent(
tool_name=context.call.tool_name, content="⌨️ 已向后台进程发送输入"
)
)
return ToolResult(
output=f"已发送按键。📺 屏幕刷新后快照如下:\n```text\n{output}\n```"
)
@tool(
name="read_sandbox_screen",
description=(
"主动窥探并读取当前虚拟屏幕的画面。当你觉得后台程序可能已经渲染出新内容时,可以使用此工具。"
),
)
async def read_sandbox_screen(self, context: RunContext) -> ToolResult:
interactive_session = self._get_pty(context)
if not interactive_session:
return ToolResult(output="没有运行中的虚拟屏幕。").as_error()
output = await interactive_session.read_output()
return ToolResult(output=f"📺 当前屏幕快照:\n```text\n{output}\n```")
@tool(
name="interrupt_sandbox",
description=(
"向当前沙箱发送 Ctrl+C (SIGINT) 信号,强制中断正在死循环或挂起的后台进程。"
),
)
async def interrupt_sandbox(self, context: RunContext) -> ToolResult:
interactive_session = self._get_pty(context)
if not interactive_session:
return ToolResult(output="没有运行中的虚拟屏幕需要中断。").as_error()
await interactive_session.interrupt()
await asyncio.sleep(1)
output = await interactive_session.read_output()
if context and context.run.event_bus:
await context.run.event_bus.emit(
ToolStreamChunkEvent(
tool_name=context.call.tool_name, content="🛑 已强制中断后台进程"
)
)
return ToolResult(
output="✅ 成功发送 Ctrl+C 中断信号。"
f"📺 当前屏幕快照:\n```text\n{output}\n```"
)
@tool(
name="write_sandbox_file",
description="将文本内容写入沙箱文件系统中,支持保存大块数据或配置,避免超过对话上下文。",
)
@silent()
async def write_sandbox_file(
self, path: str, content: str, context: RunContext, sandbox: Inject.Sandbox
) -> ToolResult:
executor = await self._get_session(context, sandbox)
success = await executor.write(path, content.encode("utf-8"))
if success:
logger.info(f"📝 已向沙箱写入文件: {path}")
return ToolResult(output=f"成功将内容写入文件: {path}")
else:
from zhenxun.services.ai.core.exceptions import AbortException
raise AbortException(
reason="写入文件失败 (当前沙箱环境失联或不支持持久化IO)",
display="❌ 写入文件失败:沙箱失联",
)
@tool(
name="read_sandbox_file",
description="从沙箱文件系统中读取指定文件的文本内容。",
)
@silent()
async def read_sandbox_file(
self, path: str, context: RunContext, sandbox: Inject.Sandbox
) -> ToolResult:
executor = await self._get_session(context, sandbox)
try:
content_bytes = await executor.read(path)
content = content_bytes.decode("utf-8", errors="replace")
if content.startswith("Error:") or content.startswith("Failed to"):
return ToolResult(output=content).as_error()
logger.info(f"已读取沙箱文件 {path} (共 {len(content)} 字符)")
return ToolResult(output=content)
except Exception as e:
from zhenxun.services.ai.core.exceptions import AbortException
raise AbortException(reason=f"读取文件发生框架级异常: {e}")
@@ -0,0 +1,368 @@
import asyncio
from contextlib import asynccontextmanager
import time
from typing import Any, Literal
from pydantic import Field, create_model
from zhenxun.services.ai.context.memory.manager import memory_manager
from zhenxun.services.ai.context.memory.models import MemoryConfig
from zhenxun.services.ai.context.memory.types import (
MemorySlot,
SessionMetadata,
)
from zhenxun.services.ai.run.context import RunContext
from zhenxun.services.ai.tools.core.decorators import tool
from zhenxun.services.ai.tools.core.tool import BaseTool
from zhenxun.services.ai.tools.core.toolkit import BaseToolkit
from zhenxun.services.ai.tools.models import ToolOptions, ToolResult
_SLOT_LOCKS: dict[str, asyncio.Lock] = {}
_GLOBAL_LOCK = asyncio.Lock()
@asynccontextmanager
async def _slot_lock(session_id: str, label: str):
key = f"{session_id}:{label}"
async with _GLOBAL_LOCK:
if key not in _SLOT_LOCKS:
_SLOT_LOCKS[key] = asyncio.Lock()
lock = _SLOT_LOCKS[key]
async with lock:
yield
class MemorySlotToolkit(BaseToolkit):
"""
中期记忆槽工具箱。
向大模型开放直接编辑上下文 XML 节点的能力。
"""
default_prefix = ""
class Config:
"""声明式配置:该工具箱下的所有工具默认静默"""
shared_options = ToolOptions(silent=True)
default_instructions = """\
## 📋 状态与规则面板 (Memory Slots / 中期记忆)
该系统是你的「桌面便利贴」或「共享黑板」,用于保存你当前需要随时查阅的核心状态与全局规范。
### 💡 核心机制
- 被保存在记忆槽中的内容(如果已置顶),会在每次对话时**直接注入到你的上下文提示词中**,你无需任何搜索即可看见。
- 槽位容量极其有限,仅用于维持当前最新的运行状态。
### 📝 何时使用记忆槽?
- **维护全局规则**:例如设定"用户整体偏好"、"沟通口吻"、"全局指导原则"等需要时刻遵守的规范(使用 `update_slot`)。
- **追踪当前进度**:例如记录"待办事项清单"、"当前任务进度"、"上下文摘要"(使用 `append_slot` 列表或 `update_slot` 覆盖)。
### ⚙️ 操作规范
1. **探索可用面板**:接手新任务时,可使用 `list_slots` 宏观查看当前存在哪些状态面板。
2. **保持精简**:槽位有严格的字符数限制。当内容过长时,请主动将其归档到长期记忆后,重新提炼并覆盖槽位,或直接调用 `delete_slot` 删除不再需要的槽位。\
""" # noqa: E501
def __init__(
self,
memory_config: MemoryConfig | None = None,
namespace: str | None = None,
**kwargs: Any,
):
"""
初始化中期记忆槽工具箱。
参数:
memory_config: 记忆系统的全局配置对象,为空则使用全局默认。
namespace: 当前隔离环境的命名空间。
kwargs: 其他透传给 BaseToolkit 的参数。
"""
super().__init__(**kwargs)
self.memory_config = memory_config
self._namespace = namespace
def _get_runtime_meta_and_ctx(
self, context: RunContext, scope_name: str | None = None
) -> tuple[Any, SessionMetadata]:
ns = self._namespace or getattr(context.session, "namespace", "global")
slot_ctx = memory_manager.get_slot_context(self.memory_config, namespace=ns)
scope_builder = None
if (
self.memory_config
and self.memory_config.slots
and self.memory_config.slots.scopes
):
scopes_dict = self.memory_config.slots.scopes
if not scope_name:
scope_builder = next(iter(scopes_dict.values()))
else:
scope_builder = scopes_dict.get(scope_name)
if not scope_builder:
scope_builder = getattr(self.memory_config, "base_isolation", None)
if not scope_builder:
from zhenxun.services.ai.context.memory.types import Isolation
scope_builder = Isolation.AGENT_USER()
selector = scope_builder.resolve(
deps=context.deps,
prefix="",
default_namespace=ns,
default_agent=context.run.agent_name,
)
parts = selector.get_scope_parts()
all_scopes = {"/"}
current_path = ""
for part in parts:
current_path += f"/{part}"
all_scopes.add(current_path)
accessible_scopes = list(all_scopes)
accessible_scopes.sort(key=lambda x: len(x.split("/")))
scope_name_mapping = {}
if (
self.memory_config
and self.memory_config.slots
and self.memory_config.slots.scopes
):
for name, builder in self.memory_config.slots.scopes.items():
sel = builder.resolve(
deps=context.deps,
prefix="",
default_namespace=ns,
default_agent=context.run.agent_name,
)
scope_name_mapping[sel.scope_prefix] = name
session_meta = SessionMetadata(
session_id=context.session_id or "default_session",
selector=selector,
scope_prefix=selector.scope_prefix,
accessible_scopes=accessible_scopes,
scope_name_mapping=scope_name_mapping,
)
return slot_ctx, session_meta
async def get_tools(self, context: RunContext | None = None) -> dict[str, BaseTool]:
tools = await super().get_tools(context)
if (
not self.memory_config
or not self.memory_config.slots
or not self.memory_config.slots.enable
):
return tools
scopes_dict = self.memory_config.slots.scopes
if not scopes_dict:
return tools
scope_keys = tuple(scopes_dict.keys())
if len(scope_keys) > 1:
ScopeType = Literal[scope_keys]
UpdateArgs = create_model(
"UpdateSlotArgs",
label=(str, Field(..., description="槽位标签名称")),
content=(str, Field(..., description="槽位内容(全量覆写)")),
description=(str, Field(default="", description="槽位的简要说明")),
scope=(
ScopeType,
Field(..., description="选择记忆存储的隔离分区"),
),
)
AppendArgs = create_model(
"AppendSlotArgs",
label=(str, Field(..., description="槽位标签名称")),
text=(str, Field(..., description="要追加的文本内容")),
scope=(
ScopeType,
Field(..., description="选择记忆存储的隔离分区"),
),
)
ReadArgs = create_model(
"ReadSlotArgs",
label=(str, Field(..., description="槽位标签名称")),
scope=(
ScopeType,
Field(..., description="选择记忆存储的隔离分区"),
),
)
DeleteArgs = create_model(
"DeleteSlotArgs",
label=(str, Field(..., description="槽位标签名称")),
scope=(
ScopeType,
Field(..., description="选择记忆存储的隔离分区"),
),
)
ListArgs = create_model(
"ListSlotsArgs",
scope=(
ScopeType,
Field(..., description="选择记忆存储的隔离分区"),
),
)
for t_name, t in tools.items():
if t_name.endswith("update_slot"):
t.args_schema = UpdateArgs
elif t_name.endswith("append_slot"):
t.args_schema = AppendArgs
elif t_name.endswith("read_slot"):
t.args_schema = ReadArgs
elif t_name.endswith("delete_slot"):
t.args_schema = DeleteArgs
elif t_name.endswith("list_slots"):
t.args_schema = ListArgs
return tools
@tool(
description="列出当前所有可用的记忆槽(包括未置顶显示的槽位),方便你了解有哪些信息可供读取或更新。"
)
async def list_slots(self, context: RunContext, **kwargs) -> ToolResult:
scope_name = kwargs.get("scope")
slot_ctx, meta = self._get_runtime_meta_and_ctx(context, scope_name)
if not slot_ctx:
return ToolResult(output="错误:未配置记忆槽后端").as_error()
slots = await slot_ctx.list_all_slots(meta)
if not slots:
return ToolResult(output="当前没有任何记忆槽。")
res = ["已创建的记忆槽列表:"]
show_scope = False
if (
self.memory_config
and self.memory_config.slots
and self.memory_config.slots.scopes
and len(self.memory_config.slots.scopes) > 1
):
show_scope = True
for s in slots:
pin_str = "置顶" if s.pinned else "隐藏"
if show_scope:
semantic_name = meta.scope_name_mapping.get(s.scope, "未知")
res.append(
f"- [{s.label}] (分区: "
f"{semantic_name}, {pin_str}) - {s.description}"
)
else:
res.append(f"- [{s.label}] ({pin_str}) - {s.description}")
return ToolResult(output="\n".join(res))
@tool(description="读取某个尚未展示在上下文中的记忆槽完整内容。")
async def read_slot(self, label: str, context: RunContext, **kwargs) -> ToolResult:
scope_name = kwargs.get("scope")
slot_ctx, meta = self._get_runtime_meta_and_ctx(context, scope_name)
if not slot_ctx:
return ToolResult(output="错误:未配置记忆槽后端").as_error()
slot = await slot_ctx.get_slot(meta, label)
if not slot:
return ToolResult(output=f"未找到标签为 '{label}' 的槽位。").as_error()
return ToolResult(output=f"[{label}] 内容:\n{slot.content}")
@tool(description=("更新或新建记忆槽的内容(全量覆写)。"))
async def update_slot(
self,
label: str,
content: str,
context: RunContext,
description: str = "",
**kwargs,
) -> ToolResult:
scope_name = kwargs.get("scope")
slot_ctx, meta = self._get_runtime_meta_and_ctx(context, scope_name)
if not slot_ctx:
return ToolResult(output="错误:未配置记忆槽后端").as_error()
target_sid = meta.scope_prefix
async with _slot_lock(target_sid, label):
slot = await slot_ctx.get_slot(meta, label)
if not slot:
slot = MemorySlot(
label=label,
content=content,
scope=meta.scope_prefix,
description=description,
)
else:
slot.content = content
slot.scope = meta.scope_prefix
if description:
slot.description = description
slot.updated_at = time.time()
if len(content) > slot.size_limit:
return ToolResult(
output=(
f"错误:内容长度超过限制 ({len(content)} > {slot.size_limit})。"
)
).as_error()
await slot_ctx.set_slot(meta, slot)
return ToolResult(output=f"已成功将 '{label}' 更新至记忆槽中。")
@tool(description="在指定记忆槽的末尾追加文本(例如追加待办事项清单)。")
async def append_slot(
self, label: str, text: str, context: RunContext, **kwargs
) -> ToolResult:
scope_name = kwargs.get("scope")
slot_ctx, meta = self._get_runtime_meta_and_ctx(context, scope_name)
if not slot_ctx:
return ToolResult(output="错误:未配置记忆槽后端").as_error()
slot = await slot_ctx.get_slot(meta, label)
if not slot:
return ToolResult(
output=(
f"错误:标签为 '{label}' 的槽位不存在,请先使用 update_slot 创建。"
)
).as_error()
target_sid = meta.scope_prefix
async with _slot_lock(target_sid, label):
slot = await slot_ctx.get_slot(meta, label)
if not slot:
return ToolResult(
output="错误:并发写入异常,槽位已被删除。"
).as_error()
sep = "\n" if slot.content and not slot.content.endswith("\n") else ""
new_content = f"{slot.content}{sep}{text}"
if len(new_content) > slot.size_limit:
return ToolResult(
output=(
"错误:追加后总长度超过限制 "
f"({len(new_content)} > {slot.size_limit})。"
)
).as_error()
slot.content = new_content
slot.updated_at = time.time()
await slot_ctx.set_slot(meta, slot)
return ToolResult(output=f"已成功追加至 '{label}'。")
@tool(description="删除不再需要的记忆槽(全量删除)。")
async def delete_slot(
self, label: str, context: RunContext, **kwargs
) -> ToolResult:
scope_name = kwargs.get("scope")
slot_ctx, meta = self._get_runtime_meta_and_ctx(context, scope_name)
if not slot_ctx:
return ToolResult(output="错误:未配置记忆槽后端").as_error()
target_sid = meta.scope_prefix
async with _slot_lock(target_sid, label):
await slot_ctx.delete_slot(meta, label, meta.scope_prefix)
return ToolResult(output=f"已成功删除槽位 '{label}'。")
@@ -0,0 +1,3 @@
from .provider import GlobalMCPProvider, MCPSource, mcp_provider
__all__ = ["GlobalMCPProvider", "MCPSource", "mcp_provider"]
@@ -0,0 +1,366 @@
from __future__ import annotations
import asyncio
from collections.abc import Callable
import json
import re
from typing import Any, Literal, cast
from pydantic import BaseModel, Field
from zhenxun.configs.path_config import DATA_PATH
from zhenxun.services.ai.core.protocols.tool import (
ToolExecutable,
ToolProvider,
ToolResolvable,
)
from zhenxun.services.ai.sandbox.models import SandboxBlueprint
from zhenxun.services.ai.tools.models import ResolvedToolPayload
from zhenxun.services.ai.tools.providers.mcp.toolkit import MCPToolkit
from zhenxun.services.log import logger
from zhenxun.utils.pydantic_compat import model_dump, model_validate, model_validator
MCP_PATH = DATA_PATH / "ai" / "mcp.json"
class MCPServerConfig(BaseModel):
transport: Literal["stdio", "sse", "streamable-http", "sandbox_proxy"] = Field(
default="stdio"
)
"""传输协议类型:stdio / sse / streamable-http / sandbox_proxy"""
url: str | None = Field(default=None)
"""远端地址(用于 sse 或 streamable-http)"""
timeout: int = Field(default=30)
"""请求超时时间(秒)"""
command: str | None = None
"""启动命令(用于 stdio 或 sandbox_proxy)"""
args: list[str] = Field(default_factory=list)
"""命令参数列表"""
env: dict[str, str] | None = None
"""启动子进程的环境变量(可选)"""
cwd: str | None = Field(default=None)
"""进程的工作目录"""
install_command: str | None = Field(default=None)
"""首次运行前的安装/预热命令"""
description: str | None = None
"""服务描述信息(用于配置可读性与展示)"""
enabled: bool = Field(default=True)
"""是否全局启用"""
admin_level: int = Field(default=0)
"""执行该服务器工具所需的群管等级"""
sandbox_blueprint: SandboxBlueprint | None = Field(default=None)
"""沙箱环境装配配置(用于 sandbox_proxy 自动处理依赖)"""
class MCPToolsConfig(BaseModel):
mcpServers: dict[str, MCPServerConfig] = Field(default_factory=dict)
class GlobalMCPProvider(ToolProvider):
def __init__(self):
self._config: MCPToolsConfig | None = None
self._toolkits: dict[str, MCPToolkit] = {}
self._discovery_lock = asyncio.Lock()
self._discovered_tools: dict[str, ToolExecutable] | None = None
self.env_provider: Callable[[Any], dict[str, str]] | None = None
self.header_provider: Callable[[Any], dict[str, str]] | None = None
self._code_registered_servers: dict[str, tuple[MCPServerConfig, bool]] = {}
def register_server(
self, name: str, config: MCPServerConfig, persist: bool = False
) -> None:
"""代码优先:动态注册 MCP 服务 (支持无痕内存级注册与热重载)"""
self._code_registered_servers[name] = (config, persist)
if self._config is not None:
if persist:
if name not in self._config.mcpServers:
self._config.mcpServers[name] = config
self._save_config()
else:
json_conf = self._config.mcpServers[name]
code_dict = model_dump(config, exclude_unset=True)
json_dict = model_dump(json_conf, exclude_unset=True)
merged_dict = {**code_dict, **json_dict}
merged_conf = model_validate(MCPServerConfig, merged_dict)
if model_dump(merged_conf) != model_dump(json_conf):
self._config.mcpServers[name] = merged_conf
self._save_config()
if name not in self._toolkits:
self._setup_toolkit(name, config)
logger.info(
f"🔄 MCP 代理 '{name}' 已完成动态热重载注册 (持久化: {persist})"
)
async def unregister_server(self, name: str) -> None:
"""动态注销 MCP 服务,关闭底层进程并从配置文件中移除"""
if name in self._code_registered_servers:
self._code_registered_servers.pop(name)
if self._config and name in self._config.mcpServers:
self._config.mcpServers.pop(name)
self._save_config()
if tk := self._toolkits.pop(name, None):
await tk.close()
logger.debug(f"🛑 MCP 代理 '{name}' 已注销并销毁所有连接")
def _save_config(self) -> None:
"""将当前配置持久化到 JSON 文件"""
if not self._config:
return
try:
with MCP_PATH.open("w", encoding="utf-8") as f:
data = model_dump(self._config, exclude_unset=True)
json.dump(data, f, ensure_ascii=False, indent=2)
except Exception as e:
logger.error(f"保存 MCP 配置失败: {e}", e=e)
async def initialize(self) -> None:
if self._config is not None:
return
if not MCP_PATH.exists():
MCP_PATH.parent.mkdir(parents=True, exist_ok=True)
self._config = MCPToolsConfig(
mcpServers={
"amap-maps": MCPServerConfig(
command="npx",
args=["-y", "@amap/amap-maps-mcp-server"],
env={"AMAP_MAPS_API_KEY": "Your_API_Key"},
enabled=False,
),
"bilibili-search": MCPServerConfig(
command="npx",
args=["bilibili-mcp"],
),
"fetch": MCPServerConfig(
command="uvx",
args=["mcp-server-fetch"],
),
"bingcn": MCPServerConfig(
command="npx",
args=["bing-cn-mcp"],
),
}
)
with MCP_PATH.open("w", encoding="utf-8") as f:
data = model_dump(self._config, exclude_unset=True)
json.dump(data, f, ensure_ascii=False, indent=2)
else:
try:
with MCP_PATH.open("r", encoding="utf-8") as f:
self._config = model_validate(MCPToolsConfig, json.load(f))
except Exception as e:
logger.error(f"加载 MCP 工具配置文件失败: {e}", e=e)
self._config = MCPToolsConfig()
config_changed = False
for name, (code_conf, persist) in self._code_registered_servers.items():
if persist:
if name not in self._config.mcpServers:
self._config.mcpServers[name] = code_conf
config_changed = True
else:
json_conf = self._config.mcpServers[name]
code_dict = model_dump(code_conf, exclude_unset=True)
json_dict = model_dump(json_conf, exclude_unset=True)
merged_dict = {**code_dict, **json_dict}
merged_conf = model_validate(MCPServerConfig, merged_dict)
if model_dump(merged_conf) != model_dump(json_conf):
self._config.mcpServers[name] = merged_conf
config_changed = True
if config_changed or not MCP_PATH.exists():
self._save_config()
for name, conf in self._config.mcpServers.items():
self._setup_toolkit(name, conf)
for name, (code_conf, persist) in self._code_registered_servers.items():
if not persist and name not in self._toolkits:
self._setup_toolkit(name, code_conf)
def _setup_toolkit(self, name: str, conf: MCPServerConfig) -> None:
"""辅助方法:装载单个 MCP Toolkit"""
clean_name = re.sub(r"[^a-zA-Z0-9]", "_", name)
prefix = f"mcp_{clean_name}_"
self._toolkits[name] = MCPToolkit(
server_name=name,
prefix=prefix,
transport=conf.transport,
command=conf.command,
args=conf.args,
url=conf.url,
env=conf.env,
cwd=conf.cwd,
install_command=conf.install_command,
timeout=conf.timeout,
admin_level=conf.admin_level,
header_provider=self.header_provider,
env_provider=self.env_provider,
sandbox_blueprint=conf.sandbox_blueprint,
)
async def shutdown(self):
for tk in self._toolkits.values():
await tk.close()
self._toolkits.clear()
async def get_tools_for_server(self, server_name: str) -> dict[str, ToolExecutable]:
return await self.discover_tools(allowed_servers=[server_name])
async def discover_tools(
self,
allowed_servers: list[str] | None = None,
excluded_servers: list[str] | None = None,
) -> dict[str, ToolExecutable]:
async with self._discovery_lock:
if not self._config:
await self.initialize()
if not self._config:
return {}
if (
self._discovered_tools is not None
and allowed_servers is None
and excluded_servers is None
):
return self._discovered_tools
servers_to_query: list[str] = []
for name, conf in self._config.mcpServers.items():
if allowed_servers and name not in allowed_servers:
continue
if excluded_servers and name in excluded_servers:
continue
if not conf.enabled and not allowed_servers:
continue
servers_to_query.append(name)
for name, (conf, persist) in self._code_registered_servers.items():
if persist:
continue
if allowed_servers and name not in allowed_servers:
continue
if excluded_servers and name in excluded_servers:
continue
if not conf.enabled and not allowed_servers:
continue
if name not in servers_to_query:
servers_to_query.append(name)
all_tools: dict[str, ToolExecutable] = {}
tasks = []
for s_name in servers_to_query:
if tk := self._toolkits.get(s_name):
tasks.append(tk.get_tools())
if tasks:
results = await asyncio.gather(*tasks, return_exceptions=True)
for res in results:
if isinstance(res, dict):
for name, t in res.items():
all_tools[name] = t
elif isinstance(res, list):
for t in res:
all_tools[t.name] = t
elif isinstance(res, Exception):
logger.error(f"MCP 并发发现工具失败: {res}")
if allowed_servers is None and excluded_servers is None:
self._discovered_tools = all_tools
return all_tools
async def get_tool_executable(
self, name: str, config: dict[str, Any]
) -> ToolExecutable | None:
_ = name, config
return None
class MCPSource(BaseModel):
"""显式定义的 MCP 工具源 (动态解析器)"""
server_name: str | None = None
"""目标 MCP 服务名(走全局注册表模式时使用)"""
namespace: str | None = None
"""可选命名空间标识(用于上层路由或分组)"""
config: MCPServerConfig | None = None
"""内联临时配置 (MCPServerConfig 实例)"""
fetch_all_enabled: bool = Field(default=False)
"""内部标识:是否拉取所有已启用的 MCP 服务"""
exclude_servers: list[str] | None = Field(default=None)
"""排除的服务名列表(仅当 fetch_all_enabled 为 True 时生效)"""
def __hash__(self):
return hash((self.server_name, self.namespace))
@classmethod
def all_enabled(cls, exclude: list[str] | None = None) -> "MCPSource":
"""
声明式获取所有已启用的 MCP 服务器下的工具。
参数:
exclude: 需要显式排除的 MCP 服务名称列表。
"""
return cls(fetch_all_enabled=True, exclude_servers=exclude)
@model_validator(mode="after")
def validate_source(self) -> "MCPSource":
if not self.server_name and not self.config and not self.fetch_all_enabled:
raise ValueError(
"MCPSource 必须提供 server_name 或 config,或者开启 fetch_all_enabled"
)
return self
async def resolve(self, context: Any | None = None) -> Any:
tools = []
if self.fetch_all_enabled:
server_tools_dict = await mcp_provider.discover_tools(
excluded_servers=self.exclude_servers
)
for t in server_tools_dict.values():
p = await cast(ToolResolvable, t).resolve(context)
if p:
tools.extend(p.tools)
return ResolvedToolPayload(tools=tools)
elif self.config:
import uuid
temp_server_name = self.server_name or f"inline_mcp_{uuid.uuid4().hex[:8]}"
inline_toolkit = MCPToolkit(
server_name=temp_server_name,
transport=self.config.transport,
command=self.config.command,
args=self.config.args,
url=self.config.url,
env=self.config.env,
cwd=self.config.cwd,
install_command=self.config.install_command,
timeout=self.config.timeout,
admin_level=self.config.admin_level,
sandbox_blueprint=self.config.sandbox_blueprint,
)
payload = await inline_toolkit.resolve(context)
payload.toolkits.insert(0, inline_toolkit)
return payload
else:
assert self.server_name is not None
server_tools = await mcp_provider.get_tools_for_server(self.server_name)
for t in server_tools.values():
p = await cast(ToolResolvable, t).resolve(context)
if p:
tools.extend(p.tools)
return ResolvedToolPayload(tools=tools)
mcp_provider = GlobalMCPProvider()
__all__ = ["GlobalMCPProvider", "MCPSource", "mcp_provider"]
@@ -0,0 +1,612 @@
import asyncio
import base64
from collections.abc import AsyncGenerator, Callable
from contextlib import AsyncExitStack, asynccontextmanager
import re
from typing import Any, Literal, cast
import anyio
from anyio.streams.memory import MemoryObjectReceiveStream, MemoryObjectSendStream
import httpx
from mcp import ClientSession # type: ignore
from mcp.client.sse import sse_client
from mcp.client.stdio import StdioServerParameters, stdio_client
from mcp.client.streamable_http import streamable_http_client
from mcp.shared.message import SessionMessage
from pydantic import ValidationError
from zhenxun.services.ai.core.models import ToolDefinition
from zhenxun.services.ai.core.stream_events import UserCustomEvent
from zhenxun.services.ai.run import RunContext
from zhenxun.services.ai.sandbox.addons.base import BaseMcpProxyExtension
from zhenxun.services.ai.sandbox.models import SandboxBlueprint
from zhenxun.services.ai.tools.core.tool import BaseTool
from zhenxun.services.ai.tools.core.toolkit import BaseToolkit
from zhenxun.services.ai.tools.models import ToolkitConfig, ToolResult
from zhenxun.services.log import logger
from zhenxun.utils.lifespan import LifespanManager
from zhenxun.utils.pydantic_compat import model_dump
_MESSAGE_START_CHARS = {"{", "["}
_LITERAL_PREFIXES: tuple[str, ...] = ("true", "false", "null")
def _should_ignore_exception(exc: Exception) -> bool:
"""
判断该异常是否是由非 JSON 的脏数据标准输出引起的。
如果是脏数据,则可以安全忽略。
"""
if not isinstance(exc, ValidationError):
return False
errors = exc.errors()
first = next(iter(errors), None)
if not first or first.get("type") != "json_invalid":
return False
input_value = first.get("input")
if not isinstance(input_value, str):
return False
stripped = input_value.strip()
if not stripped:
return True
first_char = stripped[0]
lowered = stripped.lower()
if first_char in _MESSAGE_START_CHARS or any(
lowered.startswith(prefix) for prefix in _LITERAL_PREFIXES
):
return False
return True
@asynccontextmanager
async def filtered_stdio_client(
server_name: str, server: StdioServerParameters
) -> AsyncGenerator[
tuple[
MemoryObjectReceiveStream[SessionMessage | Exception],
MemoryObjectSendStream[SessionMessage],
],
None,
]:
"""
包裹官方的 stdio_client,拦截并过滤掉非 JSON 格式的标准输出噪音。
"""
async with stdio_client(server=server) as (read_stream, write_stream):
filtered_send, filtered_recv = anyio.create_memory_object_stream[
SessionMessage | Exception
](0)
async def _forward_stdout() -> None:
try:
async with read_stream:
async for item in read_stream:
if isinstance(item, Exception) and _should_ignore_exception(
item
):
if isinstance(item, ValidationError):
err_input = item.errors()[0].get("input", "")
logger.debug(
f"🔇 [MCP Stdout Filter] {server_name} 忽略脏数据: "
f"{str(err_input)[:100].strip()}..."
)
continue
await filtered_send.send(item)
except anyio.ClosedResourceError:
pass
finally:
await filtered_send.aclose()
async with anyio.create_task_group() as tg:
tg.start_soon(_forward_stdout)
try:
yield filtered_recv, write_stream
finally:
tg.cancel_scope.cancel()
class MCPRemoteTool(BaseTool):
"""远端 MCP 工具在本地的代理对象"""
def __init__(
self,
name: str,
original_tool_name: str,
description: str,
parameters: dict,
toolkit: "MCPToolkit",
):
super().__init__(name=name, description=description)
self.original_tool_name = original_tool_name
self.parameters = parameters
self.toolkit = toolkit
self.parent_toolkit = toolkit
self.args_schema = None
self.metadata = (
{"admin_level": toolkit.admin_level} if toolkit.admin_level > 0 else {}
)
@property
def effective_ttl(self) -> float:
"""动态获取当前最新的 TTL 配置,支持无缝热重载"""
try:
from zhenxun.services.ai.config import get_llm_config
mcp_ttl = get_llm_config().agent_settings.mcp_cleanup_timeout
return 31536000.0 if mcp_ttl <= 0 else float(mcp_ttl)
except Exception:
return 31536000.0 if self.toolkit.ttl <= 0 else float(self.toolkit.ttl)
async def get_definition(
self, context: RunContext | None = None
) -> ToolDefinition | None:
if hasattr(self, "_dynamic_def") and self._dynamic_def is not None:
return self._dynamic_def
tool_def = ToolDefinition(
name=self.name,
description=self.description,
parameters=self.parameters,
metadata=self.metadata or {},
)
if context and self.settings.capabilities:
from zhenxun.services.ai.capabilities import (
CombinedCapability,
)
combined_cap = CombinedCapability(self.settings.capabilities)
defs = await combined_cap.prepare_tools(context, [tool_def])
if not defs:
return None
tool_def = defs[0]
return tool_def
async def execute(self, context: RunContext | None = None, **kwargs) -> ToolResult:
max_retries = 3
last_error = None
for attempt in range(max_retries):
await self.toolkit.lifespan_manager.touch(
self.toolkit.server_name, self.toolkit.effective_ttl
)
session = await self.toolkit.get_session(context)
if not session:
return ToolResult(output="MCP 连接错误").as_error()
try:
result = await session.call_tool(self.original_tool_name, kwargs)
break
except Exception as e:
last_error = e
logger.warning(
f"MCP 工具 '{self.name}' 执行失败 "
f"(尝试 {attempt + 1}/{max_retries}): {e}。"
"正在尝试自动重连自愈..."
)
if attempt < max_retries - 1:
await self.toolkit.close()
await asyncio.sleep(2**attempt)
else:
logger.error(
f"MCP 工具 '{self.name}' "
f"在重试 {max_retries} 次后仍然执行失败: {last_error}"
)
return ToolResult(output=f"MCP 错误: {last_error}").as_error()
if result.isError:
return ToolResult(output=str(result.content)).as_error()
from zhenxun.services.ai.core.messages import ImagePart, TextPart
output_content = []
img_count = 0
for item in result.content:
item_type = getattr(item, "type", "text")
if item_type == "image":
b64_data = getattr(item, "data", "")
mime_type = getattr(item, "mimeType", "image/png")
if b64_data:
try:
img_bytes = base64.b64decode(b64_data)
output_content.append(
ImagePart(raw=img_bytes, mime_type=mime_type)
)
img_count += 1
continue
except Exception as e:
logger.warning(f"MCP 图片 Base64 解码失败: {e}")
output_content.append(TextPart(text="[图片解码失败]"))
elif item_type == "text":
text = getattr(item, "text", str(item))
output_content.append(TextPart(text=text))
md_images = re.findall(r"!\[.*?\]\((https?://[^\)]+)\)", text)
for img_url in md_images:
output_content.append(ImagePart(url=img_url))
img_count += 1
else:
dumped = model_dump(item) if hasattr(item, "model_dump") else str(item)
output_content.append(TextPart(text=str(dumped)))
tool_result = ToolResult(output=output_content)
logger.info(
f"获取到 {len(output_content)} 条返回数据,提取了 {img_count} 张图片"
)
if img_count > 0 and context and context.run.event_bus:
await context.run.event_bus.emit(UserCustomEvent(display=output_content))
return tool_result
class MCPToolkit(BaseToolkit):
"""模型上下文协议 (MCP) 的工具箱封装 (支持声明式挂载与动态隔离)"""
def __init__(
self,
server_name: str,
prefix: str | None = None,
transport: Literal[
"stdio", "sse", "streamable-http", "sandbox_proxy"
] = "stdio",
command: str | None = None,
args: list[str] | None = None,
url: str | None = None,
env: dict | None = None,
cwd: str | None = None,
install_command: str | None = None,
isolation: Literal["shared", "per_session"] = "shared",
timeout: int = 30,
admin_level: int = 0,
header_provider: Callable[[RunContext], dict[str, str]] | None = None,
env_provider: Callable[[RunContext], dict[str, str]] | None = None,
ttl: int = 600,
sandbox_session_id: str | None = None,
sandbox_blueprint: SandboxBlueprint | None = None,
):
super().__init__(
config=ToolkitConfig(prefix=prefix) if prefix is not None else None
)
self.server_name = server_name
self.transport = transport
self.command = command
self.args = args or []
self.url = url
self.env = env or {}
self.cwd = cwd
self.install_command = install_command
self.timeout = timeout
self.admin_level = admin_level
self.header_provider = header_provider
self.env_provider = env_provider
self.sandbox_session_id = sandbox_session_id
self.sandbox_blueprint = sandbox_blueprint
self.ttl = ttl
self._shared_session: ClientSession | None = None
self._tools: list[BaseTool] = []
self._is_initialized = False
self.lifespan_manager = LifespanManager()
self._shared_task: asyncio.Task | None = None
self._stop_event: asyncio.Event = asyncio.Event()
self._ready_event: asyncio.Event = asyncio.Event()
self._init_exception: Exception | None = None
@property
def effective_ttl(self) -> float:
"""动态获取当前最新的 TTL 配置,支持无缝热重载"""
try:
from zhenxun.services.ai.config import get_llm_config
mcp_ttl = get_llm_config().agent_settings.mcp_cleanup_timeout
return 31536000.0 if mcp_ttl <= 0 else float(mcp_ttl)
except Exception:
return 31536000.0 if self.ttl <= 0 else float(self.ttl)
async def _run_install_command(self, force: bool = False):
"""执行环境预安装(热加载依赖)"""
if not self.install_command or not self.cwd:
return
from pathlib import Path
marker_file = Path(self.cwd) / ".zx_installed"
heuristic_missing = False
if (
"npm" in self.install_command
and not (Path(self.cwd) / "node_modules").exists()
):
heuristic_missing = True
if not force and not heuristic_missing and marker_file.exists():
return
reason = "强制触发自愈重装" if force else "首次启动或检测到环境缺失"
logger.info(
f"🔧 [{self.server_name}] {reason},正在执行: `{self.install_command}` ..."
)
try:
proc = await asyncio.create_subprocess_shell(
self.install_command,
cwd=self.cwd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
stdout, stderr = await proc.communicate()
if proc.returncode != 0:
err_msg = (
stderr.decode("utf-8", errors="ignore")
if stderr
else stdout.decode("utf-8", errors="ignore")
)
logger.error(
f"❌ [{self.server_name}] 环境安装失败!\n错误输出:\n{err_msg}"
)
raise RuntimeError(f"安装命令执行失败: {self.install_command}")
else:
marker_file.touch()
logger.info(f"✅ [{self.server_name}] 环境装配成功!")
except Exception as e:
logger.error(f"执行预热安装异常: {e}")
raise e
async def get_session(
self, context: RunContext | None = None
) -> ClientSession | None:
await self.lifespan_manager.register(
self.server_name,
ttl=self.effective_ttl,
cleanup_callback=self.release_resource,
)
if not self._is_initialized:
await self.initialize(context)
return self._shared_session
async def _spawn_session_task(self, dynamic_headers: dict, dynamic_env: dict):
"""生成后台连接任务"""
max_attempts = 2 if (self.transport == "stdio" and self.install_command) else 1
try:
for attempt in range(1, max_attempts + 1):
try:
if self.transport == "stdio":
await self._run_install_command(force=(attempt > 1))
async with AsyncExitStack() as stack:
read_stream: Any = None
write_stream: Any = None
if self.transport == "stdio":
if not self.command:
raise ValueError("stdio requires 'command'")
params = StdioServerParameters(
command=self.command,
args=self.args,
env=dynamic_env,
cwd=self.cwd,
)
transport_ctx = filtered_stdio_client(
self.server_name, params
)
elif self.transport == "sse":
if not self.url:
raise ValueError("sse requires 'url'")
transport_ctx = sse_client(
url=self.url,
headers=dynamic_headers,
timeout=self.timeout,
)
elif self.transport == "streamable-http":
if not self.url:
raise ValueError("streamable-http requires 'url'")
http_client = httpx.AsyncClient(
headers=dynamic_headers, timeout=self.timeout
)
await stack.enter_async_context(http_client)
transport_ctx = streamable_http_client(
url=self.url, http_client=http_client
)
elif self.transport == "sandbox_proxy":
from zhenxun.services.ai.sandbox.manager import (
sandbox_manager,
)
target_session_id = (
self.sandbox_session_id or "mcp_global_session"
)
bp = self.sandbox_blueprint or SandboxBlueprint()
bp.enable_network = True
driver = await sandbox_manager.get_or_create_session(
target_session_id,
blueprint=bp,
)
plugin_name = "universal_mcp"
await driver.mount_extension(plugin_name)
mcp_plugin = cast(
BaseMcpProxyExtension, driver.get_extension(plugin_name)
)
if not self.command:
raise ValueError("sandbox_proxy requires 'command'")
streams = await stack.enter_async_context(
mcp_plugin.connect_mcp(
self.command, self.args, dynamic_env
)
)
read_stream, write_stream = streams[0], streams[1]
transport_ctx = None
else:
raise ValueError(f"Unknown transport: {self.transport}")
if transport_ctx:
transport = await stack.enter_async_context(transport_ctx)
read_stream, write_stream = transport[0], transport[1]
client_session = await stack.enter_async_context(
ClientSession(read_stream, write_stream)
)
await client_session.initialize()
self._shared_session = client_session
if not self._is_initialized:
mcp_tools_res = await client_session.list_tools()
tools_list = list(mcp_tools_res.tools)
cursor = getattr(mcp_tools_res, "nextCursor", None)
while cursor:
mcp_tools_res = await client_session.list_tools(
cursor=cursor
)
tools_list.extend(mcp_tools_res.tools)
cursor = getattr(mcp_tools_res, "nextCursor", None)
for t in tools_list:
t_name = t.name.replace("-", "_")
final_name = (
f"{self.config.prefix}{t_name}"
if self.config.prefix
else t_name
)
self._tools.append(
MCPRemoteTool(
name=final_name,
original_tool_name=t.name,
description=t.description or "",
parameters=t.inputSchema,
toolkit=self,
)
)
self._is_initialized = True
self._ready_event.set()
logger.info(
f"成功连接 MCP 服务器: {self.server_name}, "
f"获取了 {len(self._tools)} 个工具"
)
await self._stop_event.wait()
return
except Exception as e:
from zhenxun.services.ai.core.exceptions import SandboxFatalError
if isinstance(e, SandboxFatalError):
raise e
if attempt < max_attempts:
logger.warning(
f"⚠️ [{self.server_name}] 进程启动或运行异常崩溃,"
"疑似环境损坏。触发自愈机制 (准备重试)..."
)
self._init_exception = e
self._shared_session = None
if not self._is_initialized:
self._is_initialized = False
continue
raise e
except Exception as e:
self._init_exception = e
logger.error(
f"MCP 服务器 '{self.server_name}' "
f"初始化连接失败(模式: {self.transport})。"
f"错误原因: {e}"
)
if "Connection closed" in str(e):
logger.error(
"提示:若是使用沙箱隧道,请进入 Docker Desktop 检查容器的日志。"
"多半是因为 npx 命令执行出错"
"(例如:镜像中缺少 Node 环境或国内网络无法连接 npm 等)。"
)
finally:
if not self._is_initialized:
self._is_initialized = False
self._shared_session = None
self._ready_event.set()
async def initialize(self, context: RunContext | None = None):
if self._is_initialized:
return
self._tools.clear()
self._stop_event.clear()
self._ready_event.clear()
self._init_exception = None
dynamic_headers = {}
dynamic_env = self.env.copy()
self._shared_task = asyncio.create_task(
self._spawn_session_task(dynamic_headers, dynamic_env)
)
await self._ready_event.wait()
if self._init_exception:
raise self._init_exception
async def get_tools(self, context: RunContext | None = None) -> dict[str, BaseTool]:
await self.lifespan_manager.register(
self.server_name,
ttl=self.effective_ttl,
cleanup_callback=self.release_resource,
)
if not self._is_initialized:
await self.initialize(context)
tools_dict = {}
for t in self._tools:
tools_dict[t.name] = t
return tools_dict
async def release_resource(self, resource_id: str):
await self.close()
async def close(self):
current_task = asyncio.current_task()
if (
self.lifespan_manager._watchdog_task
and self.lifespan_manager._watchdog_task is not current_task
):
await self.lifespan_manager.stop()
self._stop_event.set()
self._is_initialized = False
self._shared_session = None
self._tools.clear()
if (
self._shared_task
and not self._shared_task.done()
and self._shared_task is not current_task
):
try:
await asyncio.wait_for(self._shared_task, timeout=5.0)
except asyncio.TimeoutError:
logger.warning(
f"MCP 服务器 '{self.server_name}' 的任务执行超时,正在取消。"
)
self._shared_task.cancel()
self._shared_task = None
@@ -0,0 +1,21 @@
from .manager import skill_manager
from .models import (
INSTRUCTIONS,
METADATA,
RESOURCES,
Skill,
SkillFrontmatter,
SkillSource,
)
from .toolkit import SkillMetaToolkit
__all__ = [
"INSTRUCTIONS",
"METADATA",
"RESOURCES",
"Skill",
"SkillFrontmatter",
"SkillMetaToolkit",
"SkillSource",
"skill_manager",
]
@@ -0,0 +1,33 @@
from collections.abc import Sequence
from pathlib import Path
from typing import Any
from zhenxun.services.ai.capabilities import AbstractCapability
from zhenxun.services.ai.run import RunContext
from zhenxun.services.ai.tools.providers.skills.manager import skill_manager
from zhenxun.services.ai.tools.providers.skills.models import Skill, SkillSource
from zhenxun.services.ai.tools.providers.skills.toolkit import SkillMetaToolkit
from zhenxun.utils.utils import infer_plugin_namespace
class SkillCapability(AbstractCapability):
"""技能库挂载能力组件"""
def __init__(
self,
skills: Sequence[str | Path | Skill | SkillSource] | None = None,
namespace: str | None = None,
):
self.skills = skills or []
self.namespace = namespace or infer_plugin_namespace()
async def get_tools(self, context: RunContext) -> list[Any]:
tools = []
if self.skills:
resolved_skills = await skill_manager.resolve_mixed_skills(
self.skills, namespace=self.namespace
)
tools.append(SkillMetaToolkit(allowed_skills=resolved_skills))
return tools
@@ -0,0 +1,401 @@
from abc import ABC, abstractmethod
import asyncio
from collections.abc import Sequence
import json
from pathlib import Path
import re
from typing import Any, cast
from typing_extensions import Self
import aiofiles
import yaml
from zhenxun.configs.path_config import DATA_PATH
from zhenxun.services.ai.tools.providers.skills.models import (
INSTRUCTIONS,
METADATA,
RESOURCES,
Skill,
SkillBlueprintBuilder,
SkillEnvConfig,
SkillFrontmatter,
)
from zhenxun.services.log import logger
from zhenxun.utils.pydantic_compat import model_dump, model_validate
from zhenxun.utils.utils import infer_plugin_namespace
class SkillConfigManager:
"""基于 Skill ID 隔离的持久化配置金库管理器"""
def __init__(self):
self.config_path = DATA_PATH / "ai" / "skill_envs.json"
self.config = SkillEnvConfig()
self._load_sync()
def _load_sync(self):
if self.config_path.exists():
try:
with open(self.config_path, encoding="utf-8") as f:
self.config = model_validate(SkillEnvConfig, json.load(f))
except Exception as e:
logger.error(f"加载 skill_envs.json 失败: {e}")
async def save(self):
try:
self.config_path.parent.mkdir(parents=True, exist_ok=True)
async with aiofiles.open(self.config_path, "w", encoding="utf-8") as f:
content = json.dumps(
model_dump(self.config), ensure_ascii=False, indent=4
)
await f.write(content)
except Exception as e:
logger.error(f"保存 skill_envs.json 失败: {e}")
def get_envs_for_skill(self, namespace: str, skill_id: str) -> dict[str, str]:
return self.config.envs.get(namespace, {}).get(skill_id, {})
async def ensure_template(
self, namespace: str, skill_id: str, required_envs: list[str]
):
"""确保配置文件中存在所需的模板,缺失则补充为空字符串"""
if not required_envs:
return
changed = False
skill_envs = self.config.envs.setdefault(namespace, {}).setdefault(skill_id, {})
for env_key in required_envs:
if env_key not in skill_envs:
skill_envs[env_key] = ""
changed = True
if changed:
await self.save()
skill_env_manager = SkillConfigManager()
class SkillManager:
"""全局技能注册与发现中心 (本地文件系统模式)"""
_instance: "SkillManager | None" = None
def __new__(cls) -> Self:
if cls._instance is None:
cls._instance = super().__new__(cls)
return cast(Self, cls._instance)
def __init__(self):
if getattr(self, "_initialized", False):
return
self._skills: dict[str, Skill] = {}
self._scan_dirs: dict[Path, str] = {}
self._initialized = True
self._discovery_lock = asyncio.Lock()
default_skill_dir = DATA_PATH / "ai" / "skills"
default_skill_dir.mkdir(parents=True, exist_ok=True)
self._scan_dirs[default_skill_dir] = "global"
def add_scan_dir(self, path: str | Path, namespace: str | None = None):
p = Path(path).resolve()
if namespace is None:
namespace = infer_plugin_namespace()
if p not in self._scan_dirs:
self._scan_dirs[p] = namespace
def _get_valid_skill_dir(self, path: str | Path) -> Path | None:
"""统一目录校验:判断是否为合法的技能目录"""
p = Path(path).resolve()
if p.exists() and p.is_dir() and not p.name.startswith("."):
return p
return None
def _iter_valid_skill_dirs(self, target_dir: str | Path):
"""统一扫描器:迭代目标父目录下的所有合法技能子目录"""
p = Path(target_dir).resolve()
if not p.exists() or not p.is_dir():
return
for child in p.iterdir():
if valid_child := self._get_valid_skill_dir(child):
yield valid_child
async def load_local_skill(
self, path: str | Path, namespace: str | None = None
) -> Skill:
"""
[无痕加载] 读取指定路径的技能并返回独立的 Skill 实例。
该技能不会被注册到全局 manager 中,专门用于局部按需挂载。
"""
if namespace is None:
namespace = infer_plugin_namespace()
skill = await self._parse_skill_from_path(path, namespace)
if not skill:
raise ValueError(
f"无法从路径加载技能,请检查目录与 SKILL.md 是否合法: {path}"
)
return skill
async def discover_skills(self) -> dict[str, Skill]:
"""遍历所有扫描目录,发现并合并所有合法技能"""
async with self._discovery_lock:
if self._skills:
return self._skills
for scan_dir, namespace in self._scan_dirs.items():
for child in self._iter_valid_skill_dirs(scan_dir):
try:
skill = await self._parse_skill_metadata(child, namespace)
if skill:
self._skills[f"{namespace}::{skill.id}"] = skill
except Exception as e:
logger.error(f"解析技能 {child.name} 失败: {e}", e=e)
logger.info(f"技能扫描完成,共加载 {len(self._skills)} 个技能。")
return self._skills
async def get_skill_details(
self, name: str, namespace: str = "global"
) -> Skill | None:
"""获取技能详细信息,并在内存中缓存补全后的资源"""
skills = await self.discover_skills()
skill = skills.get(f"{namespace}::{name}") or skills.get(f"global::{name}")
if not skill:
return None
if skill.disclosure_level < RESOURCES:
skill = self._load_skill_resources(skill)
async with self._discovery_lock:
self._skills[f"{skill.namespace}::{skill.id}"] = skill
return skill
async def resolve_mixed_skills(
self, skills: Sequence[Any], namespace: str | None = None
) -> list[Skill]:
"""统一解析混合类型的技能列表(str, Path, Skill, SkillSource)"""
caller_namespace = namespace or infer_plugin_namespace()
resolved = []
resolvers = [self._normalize_to_resolver(s) for s in skills]
for r in resolvers:
resolved.extend(await r.resolve(self, caller_namespace))
return resolved
def _normalize_to_resolver(self, item: Any) -> "BaseSkillResolver":
if isinstance(item, Skill):
return SkillInstanceResolver(item)
elif isinstance(item, str):
return StringSkillResolver(item)
elif isinstance(item, Path):
return PathSkillResolver(item)
elif type(item).__name__ == "SkillSource":
return SourceSkillResolver(item)
raise TypeError(f"无法将对象 {type(item)} 解析为技能源。")
async def read_skill_resource(self, skill: Skill, file_path: str) -> str | None:
"""提供统一的资源文件(如 markdown 参考、脚本等)物理读取接口"""
target_file = (skill.path / file_path).resolve()
if not target_file.is_file():
fallback_file = (skill.path / "scripts" / file_path).resolve()
if fallback_file.is_file():
target_file = fallback_file
if target_file.is_file() and target_file.is_relative_to(skill.path.resolve()):
return target_file.read_text(encoding="utf-8")
return None
async def clear_cache(self):
"""清空缓存以重新扫描(方便测试热更)"""
async with self._discovery_lock:
self._skills.clear()
async def _parse_skill_from_path(
self, path: str | Path, namespace: str
) -> Skill | None:
"""按径解析技能,返回提权到最高级别的孤立技能实例,不污染全局状态"""
p = self._get_valid_skill_dir(path)
if not p:
logger.warning(
f"[SkillManager] 忽略无效技能目录 (不存在、非目录或为隐藏目录): {path}"
)
return None
try:
skill = await self._parse_skill_metadata(p, namespace)
if skill:
skill = self._load_skill_resources(skill)
return skill
except Exception as e:
logger.error(f"解析孤立技能 {p.name} 失败: {e}", e=e)
return None
async def _parse_frontmatter_and_body(
self, skill_dir: Path
) -> tuple[dict | None, str]:
skill_md_path = skill_dir / "SKILL.md"
if not skill_md_path.exists():
return None, ""
async with aiofiles.open(skill_md_path, encoding="utf-8") as f:
content = await f.read()
if not content.startswith("---"):
return None, ""
match = re.search(r"^---\s*\n(.*?)\n---\s*\n?(.*)$", content, re.DOTALL)
if not match:
return None, ""
yaml_content, body = match.groups()
try:
return yaml.safe_load(yaml_content), body.strip()
except yaml.YAMLError as e:
logger.warning(f"Skill {skill_dir.name} YAML 解析失败: {e}")
return None, ""
async def _parse_skill_metadata(
self, skill_dir: Path, namespace: str
) -> Skill | None:
fm_dict, _ = await self._parse_frontmatter_and_body(skill_dir)
if (
not isinstance(fm_dict, dict)
or "name" not in fm_dict
or "description" not in fm_dict
):
return None
blueprint, required_envs = SkillBlueprintBuilder.build(fm_dict)
fm_dict["blueprint"] = blueprint
fm_dict["required_envs"] = required_envs
skill = Skill(
frontmatter=SkillFrontmatter(**fm_dict),
path=skill_dir,
disclosure_level=METADATA,
source="local",
namespace=namespace,
)
if skill.frontmatter.required_envs:
await skill_env_manager.ensure_template(
namespace, skill.id, skill.frontmatter.required_envs
)
return skill
def _load_skill_instructions(self, skill: Skill) -> Skill:
if skill.disclosure_level >= INSTRUCTIONS:
return skill
_, body = skill_manager._parse_frontmatter_and_body_sync(skill.path)
return skill.with_disclosure_level(level=INSTRUCTIONS, instructions=body)
def _parse_frontmatter_and_body_sync(
self, skill_dir: Path
) -> tuple[dict | None, str]:
"""同步版本的回退方法"""
skill_md_path = skill_dir / "SKILL.md"
if not skill_md_path.exists():
return None, ""
content = skill_md_path.read_text(encoding="utf-8")
if not content.startswith("---"):
return None, ""
match = re.search(r"^---\s*\n(.*?)\n---\s*\n?(.*)$", content, re.DOTALL)
if not match:
return None, ""
try:
return yaml.safe_load(match.group(1)), match.group(2).strip()
except yaml.YAMLError:
return None, ""
def _load_skill_resources(self, skill: Skill) -> Skill:
if skill.disclosure_level >= RESOURCES:
return skill
if skill.disclosure_level < INSTRUCTIONS:
skill = self._load_skill_instructions(skill)
scripts = (
[
f.name
for f in (skill.path / "scripts").iterdir()
if f.is_file() and not f.name.startswith(".")
]
if (skill.path / "scripts").is_dir()
else []
)
refs = (
[
f.name
for f in (skill.path / "references").iterdir()
if f.is_file() and not f.name.startswith(".")
]
if (skill.path / "references").is_dir()
else []
)
return skill.with_disclosure_level(
level=RESOURCES, scripts=scripts, references=refs
)
skill_manager = SkillManager()
class BaseSkillResolver(ABC):
"""技能解析器基类,用于统一不同类型的技能加载逻辑"""
@abstractmethod
async def resolve(self, manager: SkillManager, namespace: str) -> list[Skill]: ...
class SkillInstanceResolver(BaseSkillResolver):
"""直接解析已有的 Skill 实例解析器"""
def __init__(self, skill: Skill):
self.skill = skill
async def resolve(self, manager: SkillManager, namespace: str) -> list[Skill]:
return [self.skill]
class StringSkillResolver(BaseSkillResolver):
"""根据技能 ID 解析技能的解析器"""
def __init__(self, skill_id: str):
self.skill_id = skill_id
async def resolve(self, manager: SkillManager, namespace: str) -> list[Skill]:
skill = await manager.get_skill_details(self.skill_id, namespace)
return [skill] if skill else []
class PathSkillResolver(BaseSkillResolver):
"""根据本地路径解析技能的解析器"""
def __init__(self, path: Path):
self.path = path
async def resolve(self, manager: SkillManager, namespace: str) -> list[Skill]:
skill = await manager.load_local_skill(self.path, namespace)
return [skill] if skill else []
class SourceSkillResolver(BaseSkillResolver):
"""根据 SkillSource 描述符动态扫描解析技能的解析器"""
def __init__(self, source: Any):
self.source = source
async def resolve(self, manager: SkillManager, namespace: str) -> list[Skill]:
candidates = []
if self.source.fetch_all:
candidates = list((await manager.discover_skills()).values())
elif self.source.scan_dir:
for child in manager._iter_valid_skill_dirs(self.source.scan_dir):
sk = await manager._parse_skill_from_path(child, namespace)
if sk:
candidates.append(sk)
resolved = []
for cand in candidates:
if (
not self.source.exclude_skills
or cand.id not in self.source.exclude_skills
):
resolved.append(cand)
return resolved
@@ -0,0 +1,260 @@
from pathlib import Path
from typing import Annotated, Any, Literal
from pydantic import BaseModel, ConfigDict, Field
from zhenxun.services.ai.sandbox.models import SandboxBlueprint
from zhenxun.utils.pydantic_compat import model_validator
DisclosureLevel = Annotated[
Literal[1, 2, 3], "Progressive disclosure levels for skill loading."
]
METADATA: DisclosureLevel = 1
INSTRUCTIONS: DisclosureLevel = 2
RESOURCES: DisclosureLevel = 3
class SkillEnvConfig(BaseModel):
"""全量技能环境变量配置根节点"""
envs: dict[str, dict[str, dict[str, str]]] = Field(default_factory=dict)
class SkillFrontmatter(BaseModel):
model_config = ConfigDict(populate_by_name=True) # type: ignore
name: str = Field(...)
"""技能名称"""
description: str = Field(...)
"""技能描述"""
compatibility: str | None = Field(default=None)
"""环境要求"""
metadata: dict[str, Any] | None = Field(default=None)
"""自定义元数据"""
allowed_tools: list[str] | None = Field(default=None, alias="allowed-tools")
"""允许的工具"""
permissions: dict[str, Any] | None = Field(default=None)
"""沙箱权限声明 (如 network: true/false)"""
blueprint: SandboxBlueprint = Field(default_factory=SandboxBlueprint)
"""统一环境装配蓝图声明 (根据 metadata 等自动推导)"""
required_envs: list[str] = Field(default_factory=list)
"""声明该技能必需的全局环境变量 Key"""
@model_validator(mode="before")
@classmethod
def parse_allowed_tools(cls, values: dict[str, Any]) -> dict[str, Any]:
"""将字符串格式的 allowed-tools 按空格切分为列表"""
key = "allowed-tools"
alt_key = "allowed_tools"
raw = values.get(key) or values.get(alt_key)
if isinstance(raw, str):
values[key] = raw.split()
return values
@property
def enable_network(self) -> bool:
"""网络权限:默认开启(高信任静态资产),
允许在 YAML 中通过 permissions.network 显式关闭"""
if self.permissions and isinstance(self.permissions, dict):
return self.permissions.get("network", True)
return True
class Skill(BaseModel):
frontmatter: SkillFrontmatter
"""解析后的YAML元数据"""
instructions: str | None = Field(default=None)
"""SKILL.md正文指令,在INSTRUCTIONS级别填充"""
path: Path
"""技能所在目录"""
disclosure_level: DisclosureLevel = Field(default=METADATA)
"""当前披露等级"""
scripts: list[str] = Field(default_factory=list)
"""脚本文件列表"""
references: list[str] = Field(default_factory=list)
"""参考文档列表"""
source: str = Field(default="local")
"""技能来源提供者标识"""
namespace: str = Field(default="global")
"""隔离所属的插件命名空间"""
def with_disclosure_level(
self,
level: DisclosureLevel,
instructions: str | None = None,
scripts: list[str] | None = None,
references: list[str] | None = None,
) -> "Skill":
"""创建一个提升了披露等级的全新 Skill 实例"""
return Skill(
frontmatter=self.frontmatter,
instructions=instructions
if instructions is not None
else self.instructions,
path=self.path,
disclosure_level=level,
scripts=scripts if scripts is not None else self.scripts,
references=references if references is not None else self.references,
source=self.source,
namespace=self.namespace,
)
@property
def id(self) -> str:
"""技能的唯一标识符(强制使用所在文件夹的名称)"""
return self.path.name
def to_xml(self) -> str:
"""将技能转化为结构化的 XML 格式,供大模型友好读取"""
xml = f"<skill>\n <name>{self.id}</name>\n <description>{self.description}</description>\n" # noqa: E501
if self.instructions:
xml += f" <instructions>\n{self.instructions}\n </instructions>\n"
if self.scripts:
xml += (
" <available_scripts>\n"
+ "\n".join(f" <script>{s}</script>" for s in self.scripts)
+ "\n </available_scripts>\n"
)
if self.references:
xml += (
" <available_references>\n"
+ "\n".join(f" <reference>{r}</reference>" for r in self.references)
+ "\n </available_references>\n"
)
xml += "</skill>"
return xml
@property
def name(self) -> str:
"""技能的显示名称(来自 YAML,可能不符合规范)"""
return self.frontmatter.name
@property
def description(self) -> str:
return self.frontmatter.description
class SkillSource(BaseModel):
"""显式定义的技能源 (动态解析器)"""
fetch_all: bool = Field(default=False)
"""是否拉取全局所有已注册的技能"""
scan_dir: Path | None = Field(default=None)
"""扫描特定物理目录下的所有技能"""
exclude_skills: list[str] | None = Field(default=None)
"""需要排除的技能 ID 列表"""
@classmethod
def all(cls, exclude: list[str] | None = None) -> "SkillSource":
"""声明式:获取系统全局挂载目录下的所有技能"""
return cls(fetch_all=True, exclude_skills=exclude)
@classmethod
def from_dir(
cls, path: str | Path, exclude: list[str] | None = None
) -> "SkillSource":
"""声明式:获取指定物理目录下的所有技能(支持私有独立技能库)"""
from pathlib import Path
return cls(scan_dir=Path(path), exclude_skills=exclude)
class SkillBlueprintBuilder:
"""环境蓝图组装器:将 YAML 字典剥离解析为沙箱 Blueprint 和环境变量要求"""
@classmethod
def build(cls, values: dict[str, Any]) -> tuple[SandboxBlueprint, list[str]]:
env_setup_data = values.get("env_setup", {})
python_packages = env_setup_data.get("python_packages", [])
system_packages = env_setup_data.get("system_packages", [])
node_packages = env_setup_data.get("node_packages", [])
bins = env_setup_data.get("bins", [])
install_scripts = env_setup_data.get("install_scripts", [])
required_envs = values.get("required_envs", [])
metadata = values.get("metadata", {})
if isinstance(metadata, str):
import json
try:
metadata = json.loads(metadata)
except Exception:
metadata = {}
if isinstance(metadata, dict):
for data in metadata.values():
if not isinstance(data, dict):
continue
requires = data.get("requires", {})
if isinstance(requires, dict):
for bin_key in ("bins", "anyBins"):
b = requires.get(bin_key, [])
if isinstance(b, list):
bins.extend(b)
envs = requires.get("env", [])
if isinstance(envs, list):
required_envs.extend(envs)
installs = data.get("install", [])
if isinstance(installs, list):
for inst in installs:
if not isinstance(inst, dict):
continue
kind = inst.get("kind", "").lower()
if kind in ("python", "pip", "uv"):
pkg = inst.get("package", "")
if isinstance(pkg, str) and pkg:
python_packages.extend(pkg.split())
elif kind in ("node", "npm"):
pkg = inst.get("package", "")
if isinstance(pkg, str) and pkg:
node_packages.append(pkg)
elif kind in ("apt"):
pkg = inst.get("formula") or inst.get("package") or ""
if isinstance(pkg, str) and pkg:
pkg_name = pkg.split("/")[-1]
system_packages.append(pkg_name)
elif kind == "go":
mod = inst.get("module", "")
if isinstance(mod, str) and mod:
install_scripts.append(f"go install {mod}")
key = "allowed-tools"
alt_key = "allowed_tools"
raw_tools = values.get(key) or values.get(alt_key)
tools_list = (
raw_tools.split()
if isinstance(raw_tools, str)
else (raw_tools if isinstance(raw_tools, list) else [])
)
import re
for tool in tools_list:
if isinstance(tool, str) and tool.startswith("Bash("):
match = re.search(r"Bash\((.*?)\)", tool)
if match:
inner = match.group(1)
cmd = inner.split(":")[0]
if cmd not in bins:
bins.append(cmd)
from zhenxun.services.ai.sandbox.models import (
AptSetup,
NodeSetup,
PythonSetup,
ShellSetup,
)
steps = []
if system_packages:
steps.append(AptSetup(packages=list(dict.fromkeys(system_packages))))
if python_packages:
steps.append(PythonSetup(packages=list(dict.fromkeys(python_packages))))
if node_packages:
steps.append(NodeSetup(packages=list(dict.fromkeys(node_packages))))
if install_scripts:
steps.append(ShellSetup(scripts=install_scripts))
blueprint = SandboxBlueprint(setup_steps=steps)
return blueprint, list(dict.fromkeys(required_envs))
@@ -0,0 +1,320 @@
from typing import Any, cast
from zhenxun.services.ai.run import Inject, RunContext
from zhenxun.services.ai.sandbox.models import SandboxBlueprint
from zhenxun.services.ai.sandbox.protocols import (
SupportsCommandExecution,
SupportsFileSystem,
)
from zhenxun.services.ai.tools.core.decorators import silent, tool
from zhenxun.services.ai.tools.core.toolkit import BaseToolkit
from zhenxun.services.ai.tools.models import ResolvedToolPayload, ToolResult
from zhenxun.services.ai.tools.providers.skills.manager import (
skill_env_manager,
skill_manager,
)
from zhenxun.services.ai.tools.providers.skills.models import Skill
from zhenxun.services.log import logger
from zhenxun.utils.pydantic_compat import model_copy
from zhenxun.utils.utils import infer_plugin_namespace
class SkillSandboxExecutionMixin:
"""技能沙箱执行与自愈逻辑混入类"""
async def _ensure_skill_workspace(
self, skill: Skill, session_id: str, sandbox: Any
) -> tuple[Any, str]:
"""负责环境隔离与沙箱启动装配,返回底层执行器和目标工作区路径"""
bp = model_copy(skill.frontmatter.blueprint, deep=True)
bp.enable_network = skill.frontmatter.enable_network
executor = await sandbox.get_or_create_session(session_id, blueprint=bp)
fs_executor = cast(SupportsFileSystem, executor)
target_workspace = f"{executor.workspace_path}/{skill.id}"
if skill.id not in executor.loaded_skills:
await fs_executor.upload_raw_dir(str(skill.path), target_workspace)
try:
await sandbox.setup_workspace_environment(session_id, target_workspace)
except Exception as e:
logger.warning(f"设置沙箱工作区环境失败: {e}")
executor.loaded_skills.add(skill.id)
return executor, target_workspace
def _prepare_skill_env_vars(
self, skill: Skill, context: RunContext | None, **kwargs
) -> dict[str, str] | ToolResult:
"""处理环境变量注入与缺失拦截"""
configured_envs = skill_env_manager.get_envs_for_skill(
skill.namespace, skill.id
)
missing_keys = [
k for k in skill.frontmatter.required_envs if not configured_envs.get(k)
]
if missing_keys:
logger.warning(f"技能 {skill.id} 因缺少环境变量 {missing_keys} 被拦截。")
return ToolResult(
output=(
f"❌ 技能执行被系统拦截:缺少必需的全局环境变量 "
f"{missing_keys}。\n"
"💡 [智能体自愈引导]:当前技能的底层配置缺失,无法正常运行。"
"请你立即停止尝试,并向用户抱歉,提示用户(或 Bot 管理员)"
"在机器人后端的 `data/ai/skill_envs.json` 文件中为该技能配置"
"相应的环境变量(API Key 等),配置完成后方可使用。"
),
).as_error()
env_vars = {}
for k, v in configured_envs.items():
if v:
env_vars[k] = str(v)
for k, v in kwargs.items():
env_vars[k] = str(v)
if context:
for k, v in context.state.items():
if isinstance(v, str | int | float | bool):
env_vars[k] = str(v)
final_env = {}
for k, v in env_vars.items():
final_env[k] = str(v)
final_env[k.upper()] = str(v)
return final_env
def _format_execution_result(self, result: Any, command: str) -> ToolResult:
"""统一处理沙箱输出并转换为 ToolResult"""
output = (
result.stdout + ("\nSTDERR:\n" + result.stderr if result.stderr else "")
).strip()
if getattr(result, "is_timeout", False) or result.exit_code == -1:
final_output = (
f"🚨 终端命令执行发生严重系统异常或超时被强杀\n"
f"(Exit Code: {result.exit_code})!\n"
"这通常意味着网络不通、下载数据过大耗时太长,"
"或沙箱环境崩溃。输出为空。"
)
elif result.exit_code != 0:
final_output = (
f"❌ 终端命令执行失败 (Exit Code: {result.exit_code})。\n"
f"输出日志:\n{output or '无日志输出'}"
)
else:
final_output = output or "✅ 执行成功 (无控制台输出)。"
logger.info(f"终端命令 {command} 执行完毕, Exit Code: {result.exit_code}")
tool_result = ToolResult(output=final_output)
if result.exit_code != 0:
tool_result = tool_result.as_error()
return tool_result
async def _execute_skill_command_in_sandbox(
self,
skill: Skill,
command: str,
context: RunContext | None,
sandbox: Any,
**kwargs,
) -> ToolResult:
"""主调度方法"""
session_id = (
context.session_id
if context and context.session_id
else f"skill_{skill.id}_session"
)
executor, target_workspace = await self._ensure_skill_workspace(
skill, session_id, sandbox
)
env_res = self._prepare_skill_env_vars(skill, context, **kwargs)
if isinstance(env_res, ToolResult):
return env_res
env_res["SKILL_DIR"] = target_workspace
cmd_executor = cast(SupportsCommandExecution, executor)
result = await cmd_executor.run_process(
command, cwd=target_workspace, timeout=180, env=env_res
)
return self._format_execution_result(result, command)
class SkillMetaToolkit(BaseToolkit, SkillSandboxExecutionMixin):
"""动态发现模式。提供通用的元工具,供大模型按需加载和执行任意可用技能。"""
default_prefix = ""
def __init__(self, allowed_skills: list[Skill] | None = None, **kwargs):
super().__init__(**kwargs)
self._allowed_skills = allowed_skills
if self._allowed_skills is not None:
self._instance_instructions = (
(self._instance_instructions or self.default_instructions)
+ "\n\n🚨 **[系统安全提示]**:当前运行在严格沙盒隔离模式,"
"你只能访问特定被授权的技能。"
)
async def _get_skill(self, skill_name: str) -> Skill | None:
"""按需从隔离白名单或全局管理器中获取技能"""
if self._allowed_skills is not None:
for s in self._allowed_skills:
if s.id == skill_name or s.name == skill_name:
return s
return None
return await skill_manager.get_skill_details(
skill_name, namespace=infer_plugin_namespace()
)
async def resolve(self, context: RunContext | None = None) -> ResolvedToolPayload:
payload = await super().resolve(context)
if self._allowed_skills is not None:
catalog_parts = []
for skill in self._allowed_skills:
catalog_parts.append(
f" <skill>\n <name>{skill.id}</name>\n"
f" <description>{skill.description}</description>\n </skill>"
)
if catalog_parts:
catalog_xml = (
"<available_skills>\n"
+ "\n".join(catalog_parts)
+ "\n</available_skills>"
)
payload.injected_prompts.append(
f"--- 当前受限的可用技能库 ---\n\n{catalog_xml}"
)
return payload
default_instructions = """\
## 技能元工具系统
你可以通过此工具箱动态加载和执行外部技能。工作流如下:
1. 使用 `read_skill_instructions` 传入技能名称,获取该技能的完整指南。
2. 系统将返回严谨的 `<skill>` XML 节点树,请仔细阅读 `<instructions>` 了解业务规则。
3. **[重点] 环境装配与执行规范**:
- **环境变量**:系统已自动将该技能的物理根目录注入为环境变量 `$SKILL_DIR`,在执行终端命令时可直接使用它来定位文件(例如 `cat $SKILL_DIR/package.json`)。
- **按需安装(懒加载)**:沙箱内通常已通过底层 Blueprint 预装了所需的依赖包。**即使技能指南(说明文档)中写了“安装依赖”的步骤,你也必须忽略它!** 严禁在没有任何报错的情况下主动去执行安装命令(如 `npm install`, `pip install`)。
- **脚本执行**:若指南中要求执行物理存在的脚本文件(如 `<available_scripts>` 节点列出的文件),请统一调用 `execute_skill_command` 并加上解释器和完整路径(如 `python3 $SKILL_DIR/scripts/xxx.py`)。
- **终端执行**:若指南中提供的是纯命令行终端指令(例如 `curl`, `infsh`, `gh` 等),请调用 `execute_skill_command` 在技能专属沙箱中直接执行该命令。
- **智能自愈 (Agentic Healing)**:如果执行脚本或命令时失败(如 Exit Code 非 0),你必须自主阅读输出日志 (Stderr/Stdout),分析报错原因。
- **只有当运行报错且明确提示缺少依赖时**(如 `command not found`, `ModuleNotFoundError` 等),你才可以调用 `execute_skill_command` 安装缺失的依赖(Python 依赖强烈建议使用 `uv pip install <pkg>`,Node 使用 `npm install <pkg>`),安装成功后再次重试。
- 如果是其他错误,请结合技能指南调整参数或操作流程后重试。
4. 如需阅读参考文档,请参考 `<available_references>` 节点并调用 `read_skill_file`。""" # noqa: E501
@tool(
name="read_skill_instructions",
description="加载指定技能的完整使用说明与可用资源清单。返回值为 XML 结构。",
)
@silent()
async def read_skill_instructions(self, skill_name: str) -> ToolResult:
skill = await self._get_skill(skill_name)
if not skill:
return ToolResult(output=f"越权操作或未找到技能: {skill_name}").as_error()
res = skill.to_xml()
res += (
"\n\n**提示**: 你可以使用 `read_skill_file` 工具读取该技能"
"目录下的任何附加文件(如 references/ 或 templates/ 下的文档)。"
)
logger.info(f"已加载技能 {skill.name} 指南。")
return ToolResult(output=res)
@tool(
name="execute_skill_command",
description=(
"在技能专属的安全沙箱内执行任意终端命令行指令"
"(如 curl, npm, gh 等)。仅当技能指南中提供的是终端命令"
"而非特定脚本文件时使用。"
),
)
async def execute_skill_command(
self,
skill_name: str,
command: str,
context: RunContext | None = None,
sandbox: Inject.Sandbox = None,
**kwargs,
) -> ToolResult:
skill = await self._get_skill(skill_name)
if not skill:
return ToolResult(
output=f"越权操作或未找到技能: {skill_name}",
).as_error()
if sandbox is None and context is not None:
sandbox = Inject._providers["sandbox"]["global"](context)
return await self._execute_skill_command_in_sandbox(
skill, command, context, sandbox, **kwargs
)
@tool(
name="read_skill_file",
description=(
"安全读取指定技能目录下的附加文件"
"(如 references/ 里的参考文档或 scripts/ 里的代码)。"
),
)
@silent()
async def read_skill_file(
self,
skill_name: str,
file_path: str,
context: RunContext | None = None,
sandbox: Inject.Sandbox = None,
) -> ToolResult:
skill = await self._get_skill(skill_name)
if not skill:
return ToolResult(output=f"越权操作或未找到技能: {skill_name}").as_error()
content = await skill_manager.read_skill_resource(skill, file_path)
if content is not None:
logger.info(
f"已从本地读取文件 {file_path} (共 {len(content)} 字符)。"
f"内容摘要: {content[:100]}..."
)
return ToolResult(output=content)
session_id = (
context.session_id
if context and context.session_id
else f"skill_{skill.id}_session"
)
bp = SandboxBlueprint(enable_network=skill.frontmatter.enable_network)
try:
if sandbox is None and context is not None:
sandbox = Inject._providers["sandbox"]["global"](context)
executor = await sandbox.get_or_create_session(session_id, blueprint=bp)
fs_executor = cast(SupportsFileSystem, executor)
clean_file_path = file_path.lstrip("/")
sandbox_target_path = (
f"{executor.workspace_path}/{skill.id}/{clean_file_path}"
)
content = await fs_executor.read_raw_file(sandbox_target_path)
if content.startswith("Error: File") or content.startswith("Failed to"):
return ToolResult(
output=(
f"❌ 找不到文件: {file_path} "
f"(Provider {skill.source} 与沙箱中均未找到)"
),
).as_error()
logger.info(
"已从沙箱读取动态生成的文件 "
f"{sandbox_target_path} (共 {len(content)} 字符)。"
)
return ToolResult(output=content)
except Exception as e:
return ToolResult(output=f"❌ 尝试读取沙箱文件时发生异常: {e}").as_error()