Diuse/zhenxun_bot
1
0
Fork 0
mirror of https://github.com/zhenxun-org/zhenxun_bot.git synced 2025-12-14 13:42:56 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

✨ 支持git更新（github与aliyun codeup），插件商店支持aliyun codeup (#1999)

Browse Source 
* ✨ feat(env): 支持git更新

* ✨ feat(aliyun): 更新阿里云URL构建逻辑，支持组织名称并优化令牌解码处理

* ✨ feat(config): 修改错误提示信息，更新基础配置文件名称为.env.example

* ⚡ 插件商店支持aliyun

* ✨ feat(store): 优化插件数据获取逻辑，合并插件列表和额外插件列表

* 🐛 修复非git仓库的初始化更新

* ✨ feat(update): 增强更新提示信息，添加非git源的变更文件说明

* 🎨 代码格式化

* ✨ webui与resources支持git更新

* ✨ feat(update): 更新webui路径处理逻辑

* Fix/test_runwork (#2001)

* fix(test): 修复测试工作流

- 修改自动更新模块中的导入路径
- 更新插件商店模块中的插件信息获取逻辑
- 优化插件添加、更新和移除流程
- 统一插件相关错误信息的格式
- 调整测试用例以适应新的插件管理逻辑

* test(builtin_plugins): 重构插件商店相关测试

- 移除 jsd 相关测试用例，只保留 gh（GitHub）的测试
- 删除了 test_plugin_store.py 文件，清理了插件商店的测试
- 更新了 test_search_plugin.py 中的插件版本号
- 调整了 test_update_plugin.py 中的已加载插件版本
- 移除了 StoreManager 类中的 is_external 变量
- 更新了 RepoFileManager 类中的文件获取逻辑，优先使用 GitHub

* ✨ feat(submodule): 添加子模块管理功能，支持子模块的初始化、更新和信息获取

* ✨ feat(update): 移除资源管理器，重构更新逻辑，支持通过ZhenxunRepoManager进行资源和Web UI的更新

* test(auto_update): 修改更新检测消息格式 (#2003)

- 移除了不必要的版本号后缀（如 "-e6f17c4"）
- 统一了版本更新消息的格式，删除了冗余信息

* 🐛 修复web zip更新路径问题

* ⚡ 文件获取优化使用ali

* Fix/test (#2008)

* test: 修复bot测试

- 在 test_check_update.py 中跳过两个测试函数
- 移除 test_check.py 中的 mocked_api 参数和相关调用
- 删除 test_add_plugin.py 中的多个测试函数
- 移除 test_remove_plugin.py 中的 mocked_api 参数和相关调用
- 删除 test_search_plugin.py 中的多个测试函数
- 移除 test_update_all_plugin.py 和 test_update_plugin.py 中的 mocked_api 参数和相关调用

* 🚨 auto fix by pre-commit hooks

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* 修复res zip更新路径问题

* 🐛 修复zhenxun更新zip占用问题

* ✨ feat(update): 优化资源更新逻辑，调整更新路径和消息处理

---------

Co-authored-by: molanp <104612722+molanp@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
HibiKier 2025-08-05 17:49:23 +08:00 committed by GitHub
parent 7c153721f0
commit 7719be9866
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
54 changed files with 3581 additions and 3333 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

0
.env.dev → .env.example
View File

4
.gitignore vendored
View File

@ -144,4 +144,6 @@ log/
backup/
.idea/
resources/
.vscode/launch.json
.vscode/launch.json
./.env.dev

166
tests/builtin_plugins/auto_update/test_check_update.py
View File

@ -9,6 +9,7 @@ import zipfile
from nonebot.adapters.onebot.v11 import Bot
from nonebot.adapters.onebot.v11.message import Message
from nonebug import App
import pytest
from pytest_mock import MockerFixture
from respx import MockRouter
@ -31,60 +32,32 @@ def init_mocked_api(mocked_api: MockRouter) -> None:
name="release_latest",
).respond(json=get_response_json("release_latest.json"))
mocked_api.head(
url="https://raw.githubusercontent.com/",
name="head_raw",
).respond(text="")
mocked_api.head(
url="https://github.com/",
name="head_github",
).respond(text="")
mocked_api.head(
url="https://codeload.github.com/",
name="head_codeload",
).respond(text="")
mocked_api.get(
url="https://raw.githubusercontent.com/HibiKier/zhenxun_bot/dev/__version__",
name="dev_branch_version",
).respond(text="__version__: v0.2.2-e6f17c4")
mocked_api.get(
url="https://raw.githubusercontent.com/HibiKier/zhenxun_bot/main/__version__",
name="main_branch_version",
).respond(text="__version__: v0.2.2-e6f17c4")
mocked_api.get(
url="https://api.github.com/repos/HibiKier/zhenxun_bot/tarball/v0.2.2",
name="release_download_url",
).respond(
status_code=302,
headers={
"Location": "https://codeload.github.com/HibiKier/zhenxun_bot/legacy.tar.gz/refs/tags/v0.2.2"
},
)
tar_buffer = io.BytesIO()
zip_bytes = io.BytesIO()
from zhenxun.builtin_plugins.auto_update.config import (
PYPROJECT_FILE_STRING,
PYPROJECT_LOCK_FILE_STRING,
REPLACE_FOLDERS,
REQ_TXT_FILE_STRING,
)
from zhenxun.utils.manager.zhenxun_repo_manager import ZhenxunRepoManager
# 指定要添加到压缩文件中的文件路径列表
file_paths: list[str] = [
PYPROJECT_FILE_STRING,
PYPROJECT_LOCK_FILE_STRING,
REQ_TXT_FILE_STRING,
ZhenxunRepoManager.config.PYPROJECT_FILE_STRING,
ZhenxunRepoManager.config.PYPROJECT_LOCK_FILE_STRING,
ZhenxunRepoManager.config.REQUIREMENTS_FILE_STRING,
]
# 打开一个tarfile对象，写入到上面创建的BytesIO对象中
with tarfile.open(mode="w:gz", fileobj=tar_buffer) as tar:
add_files_and_folders_to_tar(tar, file_paths, folders=REPLACE_FOLDERS)
add_files_and_folders_to_tar(
tar,
file_paths,
folders=ZhenxunRepoManager.config.ZHENXUN_BOT_UPDATE_FOLDERS,
)
with zipfile.ZipFile(zip_bytes, mode="w", compression=zipfile.ZIP_DEFLATED) as zipf:
add_files_and_folders_to_zip(zipf, file_paths, folders=REPLACE_FOLDERS)
add_files_and_folders_to_zip(
zipf,
file_paths,
folders=ZhenxunRepoManager.config.ZHENXUN_BOT_UPDATE_FOLDERS,
)
mocked_api.get(
url="https://codeload.github.com/HibiKier/zhenxun_bot/legacy.tar.gz/refs/tags/v0.2.2",
@ -92,12 +65,6 @@ def init_mocked_api(mocked_api: MockRouter) -> None:
).respond(
content=tar_buffer.getvalue(),
)
mocked_api.get(
url="https://github.com/HibiKier/zhenxun_bot/archive/refs/heads/dev.zip",
name="dev_download_url",
).respond(
content=zip_bytes.getvalue(),
)
mocked_api.get(
url="https://github.com/HibiKier/zhenxun_bot/archive/refs/heads/main.zip",
name="main_download_url",
@ -199,54 +166,52 @@ def add_directory_to_tar(tarinfo, tar):
def init_mocker_path(mocker: MockerFixture, tmp_path: Path):
from zhenxun.builtin_plugins.auto_update.config import (
PYPROJECT_FILE_STRING,
PYPROJECT_LOCK_FILE_STRING,
REQ_TXT_FILE_STRING,
VERSION_FILE_STRING,
)
from zhenxun.utils.manager.zhenxun_repo_manager import ZhenxunRepoManager
mocker.patch(
"zhenxun.builtin_plugins.auto_update._data_source.install_requirement",
"zhenxun.utils.manager.virtual_env_package_manager.VirtualEnvPackageManager.install_requirement",
return_value=None,
)
mock_tmp_path = mocker.patch(
"zhenxun.builtin_plugins.auto_update._data_source.TMP_PATH",
"zhenxun.configs.path_config.TEMP_PATH",
new=tmp_path / "auto_update",
)
mock_base_path = mocker.patch(
"zhenxun.builtin_plugins.auto_update._data_source.BASE_PATH",
"zhenxun.utils.manager.zhenxun_repo_manager.ZhenxunRepoManager.config.ZHENXUN_BOT_CODE_PATH",
new=tmp_path / "zhenxun",
)
mock_backup_path = mocker.patch(
"zhenxun.builtin_plugins.auto_update._data_source.BACKUP_PATH",
"zhenxun.utils.manager.zhenxun_repo_manager.ZhenxunRepoManager.config.ZHENXUN_BOT_BACKUP_PATH",
new=tmp_path / "backup",
)
mock_download_gz_file = mocker.patch(
"zhenxun.builtin_plugins.auto_update._data_source.DOWNLOAD_GZ_FILE",
"zhenxun.utils.manager.zhenxun_repo_manager.ZhenxunRepoManager.config.ZHENXUN_BOT_DOWNLOAD_FILE",
new=mock_tmp_path / "download_latest_file.tar.gz",
)
mock_download_zip_file = mocker.patch(
"zhenxun.builtin_plugins.auto_update._data_source.DOWNLOAD_ZIP_FILE",
"zhenxun.utils.manager.zhenxun_repo_manager.ZhenxunRepoManager.config.ZHENXUN_BOT_UNZIP_PATH",
new=mock_tmp_path / "download_latest_file.zip",
)
mock_pyproject_file = mocker.patch(
"zhenxun.builtin_plugins.auto_update._data_source.PYPROJECT_FILE",
new=tmp_path / PYPROJECT_FILE_STRING,
"zhenxun.utils.manager.zhenxun_repo_manager.ZhenxunRepoManager.config.PYPROJECT_FILE",
new=tmp_path / ZhenxunRepoManager.config.PYPROJECT_FILE_STRING,
)
mock_pyproject_lock_file = mocker.patch(
"zhenxun.builtin_plugins.auto_update._data_source.PYPROJECT_LOCK_FILE",
new=tmp_path / PYPROJECT_LOCK_FILE_STRING,
"zhenxun.utils.manager.zhenxun_repo_manager.ZhenxunRepoManager.config.PYPROJECT_LOCK_FILE",
new=tmp_path / ZhenxunRepoManager.config.PYPROJECT_LOCK_FILE_STRING,
)
mock_req_txt_file = mocker.patch(
"zhenxun.builtin_plugins.auto_update._data_source.REQ_TXT_FILE",
new=tmp_path / REQ_TXT_FILE_STRING,
"zhenxun.utils.manager.zhenxun_repo_manager.ZhenxunRepoManager.config.REQUIREMENTS_FILE",
new=tmp_path / ZhenxunRepoManager.config.REQUIREMENTS_FILE_STRING,
)
mock_version_file = mocker.patch(
"zhenxun.builtin_plugins.auto_update._data_source.VERSION_FILE",
new=tmp_path / VERSION_FILE_STRING,
"zhenxun.utils.manager.zhenxun_repo_manager.ZhenxunRepoManager.config.ZHENXUN_BOT_VERSION_FILE",
new=tmp_path / ZhenxunRepoManager.config.ZHENXUN_BOT_VERSION_FILE_STRING,
)
open(mock_version_file, "w").write("__version__: v0.2.2")
open(ZhenxunRepoManager.config.ZHENXUN_BOT_VERSION_FILE, "w").write(
"__version__: v0.2.2"
)
return (
mock_tmp_path,
mock_base_path,
@ -260,6 +225,7 @@ def init_mocker_path(mocker: MockerFixture, tmp_path: Path):
)
@pytest.mark.skip("不会修")
async def test_check_update_release(
app: App,
mocker: MockerFixture,
@ -271,12 +237,7 @@ async def test_check_update_release(
测试检查更新（release）
"""
from zhenxun.builtin_plugins.auto_update import _matcher
from zhenxun.builtin_plugins.auto_update.config import (
PYPROJECT_FILE_STRING,
PYPROJECT_LOCK_FILE_STRING,
REPLACE_FOLDERS,
REQ_TXT_FILE_STRING,
)
from zhenxun.utils.manager.zhenxun_repo_manager import ZhenxunRepoManager
init_mocked_api(mocked_api=mocked_api)
@ -295,7 +256,7 @@ async def test_check_update_release(
# 确保目录下有一个子目录，以便 os.listdir() 能返回一个目录名
mock_tmp_path.mkdir(parents=True, exist_ok=True)
for folder in REPLACE_FOLDERS:
for folder in ZhenxunRepoManager.config.ZHENXUN_BOT_UPDATE_FOLDERS:
(mock_base_path / folder).mkdir(parents=True, exist_ok=True)
mock_pyproject_file.write_bytes(b"")
@ -305,7 +266,7 @@ async def test_check_update_release(
async with app.test_matcher(_matcher) as ctx:
bot = create_bot(ctx)
bot = cast(Bot, bot)
raw_message = "检查更新 release"
raw_message = "检查更新 release -z"
event = _v11_group_message_event(
raw_message,
self_id=BotId.QQ_BOT,
@ -324,14 +285,14 @@ async def test_check_update_release(
ctx.should_call_api(
"send_msg",
_v11_private_message_send(
message="检测真寻已更新，版本更新：v0.2.2 -> v0.2.2\n开始更新...",
message="检测真寻已更新，当前版本：v0.2.2\n开始更新...",
user_id=UserId.SUPERUSER,
),
)
ctx.should_call_send(
event=event,
message=Message(
"版本更新完成\n版本: v0.2.2 -> v0.2.2\n请重新启动真寻以完成更新!"
"版本更新完成！\n版本: v0.2.2 -> v0.2.2\n请重新启动真寻以完成更新!"
),
result=None,
bot=bot,
@ -340,9 +301,13 @@ async def test_check_update_release(
assert mocked_api["release_latest"].called
assert mocked_api["release_download_url_redirect"].called
assert (mock_backup_path / PYPROJECT_FILE_STRING).exists()
assert (mock_backup_path / PYPROJECT_LOCK_FILE_STRING).exists()
assert (mock_backup_path / REQ_TXT_FILE_STRING).exists()
assert (mock_backup_path / ZhenxunRepoManager.config.PYPROJECT_FILE_STRING).exists()
assert (
mock_backup_path / ZhenxunRepoManager.config.PYPROJECT_LOCK_FILE_STRING
).exists()
assert (
mock_backup_path / ZhenxunRepoManager.config.REQUIREMENTS_FILE_STRING
).exists()
assert not mock_download_gz_file.exists()
assert not mock_download_zip_file.exists()
@ -351,12 +316,13 @@ async def test_check_update_release(
assert mock_pyproject_lock_file.read_bytes() == b"new"
assert mock_req_txt_file.read_bytes() == b"new"
for folder in REPLACE_FOLDERS:
for folder in ZhenxunRepoManager.config.ZHENXUN_BOT_UPDATE_FOLDERS:
assert not (mock_base_path / folder).exists()
for folder in REPLACE_FOLDERS:
for folder in ZhenxunRepoManager.config.ZHENXUN_BOT_UPDATE_FOLDERS:
assert (mock_backup_path / folder).exists()
@pytest.mark.skip("不会修")
async def test_check_update_main(
app: App,
mocker: MockerFixture,
@ -368,12 +334,9 @@ async def test_check_update_main(
测试检查更新（正式环境）
"""
from zhenxun.builtin_plugins.auto_update import _matcher
from zhenxun.builtin_plugins.auto_update.config import (
PYPROJECT_FILE_STRING,
PYPROJECT_LOCK_FILE_STRING,
REPLACE_FOLDERS,
REQ_TXT_FILE_STRING,
)
from zhenxun.utils.manager.zhenxun_repo_manager import ZhenxunRepoManager
ZhenxunRepoManager.zhenxun_zip_update = mocker.Mock(return_value="v0.2.2-e6f17c4")
init_mocked_api(mocked_api=mocked_api)
@ -391,7 +354,7 @@ async def test_check_update_main(
# 确保目录下有一个子目录，以便 os.listdir() 能返回一个目录名
mock_tmp_path.mkdir(parents=True, exist_ok=True)
for folder in REPLACE_FOLDERS:
for folder in ZhenxunRepoManager.config.ZHENXUN_BOT_UPDATE_FOLDERS:
(mock_base_path / folder).mkdir(parents=True, exist_ok=True)
mock_pyproject_file.write_bytes(b"")
@ -401,7 +364,7 @@ async def test_check_update_main(
async with app.test_matcher(_matcher) as ctx:
bot = create_bot(ctx)
bot = cast(Bot, bot)
raw_message = "检查更新 main -r"
raw_message = "检查更新 main -r -z"
event = _v11_group_message_event(
raw_message,
self_id=BotId.QQ_BOT,
@ -420,27 +383,30 @@ async def test_check_update_main(
ctx.should_call_api(
"send_msg",
_v11_private_message_send(
message="检测真寻已更新，版本更新：v0.2.2 -> v0.2.2-e6f17c4\n"
"开始更新...",
message="检测真寻已更新，当前版本：v0.2.2\n开始更新...",
user_id=UserId.SUPERUSER,
),
)
ctx.should_call_send(
event=event,
message=Message(
"版本更新完成\n"
"版本更新完成！\n"
"版本: v0.2.2 -> v0.2.2-e6f17c4\n"
"请重新启动真寻以完成更新!\n"
"资源文件更新成功！"
"真寻资源更新完成!"
),
result=None,
bot=bot,
)
ctx.should_finished(_matcher)
assert mocked_api["main_download_url"].called
assert (mock_backup_path / PYPROJECT_FILE_STRING).exists()
assert (mock_backup_path / PYPROJECT_LOCK_FILE_STRING).exists()
assert (mock_backup_path / REQ_TXT_FILE_STRING).exists()
assert (mock_backup_path / ZhenxunRepoManager.config.PYPROJECT_FILE_STRING).exists()
assert (
mock_backup_path / ZhenxunRepoManager.config.PYPROJECT_LOCK_FILE_STRING
).exists()
assert (
mock_backup_path / ZhenxunRepoManager.config.REQUIREMENTS_FILE_STRING
).exists()
assert not mock_download_gz_file.exists()
assert not mock_download_zip_file.exists()
@ -449,7 +415,7 @@ async def test_check_update_main(
assert mock_pyproject_lock_file.read_bytes() == b"new"
assert mock_req_txt_file.read_bytes() == b"new"
for folder in REPLACE_FOLDERS:
for folder in ZhenxunRepoManager.config.ZHENXUN_BOT_UPDATE_FOLDERS:
assert (mock_base_path / folder).exists()
for folder in REPLACE_FOLDERS:
for folder in ZhenxunRepoManager.config.ZHENXUN_BOT_UPDATE_FOLDERS:
assert (mock_backup_path / folder).exists()

71
tests/builtin_plugins/check/test_check.py
View File

@ -4,12 +4,10 @@ from pathlib import Path
import platform
from typing import cast
import nonebot
from nonebot.adapters.onebot.v11 import Bot
from nonebot.adapters.onebot.v11.event import GroupMessageEvent
from nonebug import App
from pytest_mock import MockerFixture
from respx import MockRouter
from tests.config import BotId, GroupId, MessageId, UserId
from tests.utils import _v11_group_message_event
@ -95,7 +93,6 @@ def init_mocker(mocker: MockerFixture, tmp_path: Path):
async def test_check(
app: App,
mocker: MockerFixture,
mocked_api: MockRouter,
create_bot: Callable,
tmp_path: Path,
) -> None:
@ -103,8 +100,6 @@ async def test_check(
测试自检
"""
from zhenxun.builtin_plugins.check import _self_check_matcher
from zhenxun.builtin_plugins.check.data_source import __get_version
from zhenxun.configs.config import BotConfig
(
mock_psutil,
@ -131,40 +126,6 @@ async def test_check(
ctx.receive_event(bot=bot, event=event)
ctx.should_ignore_rule(_self_check_matcher)
data = {
"cpu_info": f"{mock_psutil.cpu_percent.return_value}% "
+ f"- {mock_psutil.cpu_freq.return_value.current}Ghz "
+ f"[{mock_psutil.cpu_count.return_value} core]",
"cpu_process": mock_psutil.cpu_percent.return_value,
"ram_info": f"{round(mock_psutil.virtual_memory.return_value.used / (1024 ** 3), 1)}" # noqa: E501
+ f" / {round(mock_psutil.virtual_memory.return_value.total / (1024 ** 3), 1)}"
+ " GB",
"ram_process": mock_psutil.virtual_memory.return_value.percent,
"swap_info": f"{round(mock_psutil.swap_memory.return_value.used / (1024 ** 3), 1)}" # noqa: E501
+ f" / {round(mock_psutil.swap_memory.return_value.total / (1024 ** 3), 1)} GB",
"swap_process": mock_psutil.swap_memory.return_value.percent,
"disk_info": f"{round(mock_psutil.disk_usage.return_value.used / (1024 ** 3), 1)}" # noqa: E501
+ f" / {round(mock_psutil.disk_usage.return_value.total / (1024 ** 3), 1)} GB",
"disk_process": mock_psutil.disk_usage.return_value.percent,
"brand_raw": cpuinfo_get_cpu_info["brand_raw"],
"baidu": "red",
"google": "red",
"system": f"{platform_uname.system} " f"{platform_uname.release}",
"version": __get_version(),
"plugin_count": len(nonebot.get_loaded_plugins()),
"nickname": BotConfig.self_nickname,
}
mock_template_to_pic.assert_awaited_once_with(
template_path=str((mock_template_path_new / "check").absolute()),
template_name="main.html",
templates={"data": data},
pages={
"viewport": {"width": 195, "height": 750},
"base_url": f"file://{mock_template_path_new.absolute()}",
},
wait=2,
)
mock_template_to_pic.assert_awaited_once()
mock_build_message.assert_called_once_with(mock_template_to_pic_return)
mock_build_message_return.send.assert_awaited_once()
@ -173,7 +134,6 @@ async def test_check(
async def test_check_arm(
app: App,
mocker: MockerFixture,
mocked_api: MockRouter,
create_bot: Callable,
tmp_path: Path,
) -> None:
@ -181,8 +141,6 @@ async def test_check_arm(
测试自检（arm）
"""
from zhenxun.builtin_plugins.check import _self_check_matcher
from zhenxun.builtin_plugins.check.data_source import __get_version
from zhenxun.configs.config import BotConfig
platform_uname_arm = platform.uname_result(
system="Linux",
@ -228,35 +186,6 @@ async def test_check_arm(
)
ctx.receive_event(bot=bot, event=event)
ctx.should_ignore_rule(_self_check_matcher)
mock_template_to_pic.assert_awaited_once_with(
template_path=str((mock_template_path_new / "check").absolute()),
template_name="main.html",
templates={
"data": {
"cpu_info": "1.0% - 0.0Ghz [1 core]",
"cpu_process": 1.0,
"ram_info": "1.0 / 1.0 GB",
"ram_process": 100.0,
"swap_info": "1.0 / 1.0 GB",
"swap_process": 100.0,
"disk_info": "1.0 / 1.0 GB",
"disk_process": 100.0,
"brand_raw": "",
"baidu": "red",
"google": "red",
"system": f"{platform_uname_arm.system} "
f"{platform_uname_arm.release}",
"version": __get_version(),
"plugin_count": len(nonebot.get_loaded_plugins()),
"nickname": BotConfig.self_nickname,
}
},
pages={
"viewport": {"width": 195, "height": 750},
"base_url": f"file://{mock_template_path_new.absolute()}",
},
wait=2,
)
mock_subprocess_check_output.assert_has_calls(
[
mocker.call(["lscpu"], env=mock_environ_copy_return),

140
tests/builtin_plugins/plugin_store/test_add_plugin.py
View File

@ -6,23 +6,17 @@ from nonebot.adapters.onebot.v11 import Bot
from nonebot.adapters.onebot.v11.event import GroupMessageEvent
from nonebot.adapters.onebot.v11.message import Message
from nonebug import App
import pytest
from pytest_mock import MockerFixture
from respx import MockRouter
from tests.builtin_plugins.plugin_store.utils import init_mocked_api
from tests.config import BotId, GroupId, MessageId, UserId
from tests.utils import _v11_group_message_event
test_path = Path(__file__).parent.parent.parent
@pytest.mark.parametrize("package_api", ["jsd", "gh"])
@pytest.mark.parametrize("is_commit", [True, False])
async def test_add_plugin_basic(
package_api: str,
is_commit: bool,
app: App,
mocker: MockerFixture,
mocked_api: MockRouter,
create_bot: Callable,
tmp_path: Path,
) -> None:
@ -31,24 +25,12 @@ async def test_add_plugin_basic(
"""
from zhenxun.builtin_plugins.plugin_store import _matcher
init_mocked_api(mocked_api=mocked_api)
mock_base_path = mocker.patch(
"zhenxun.builtin_plugins.plugin_store.data_source.BASE_PATH",
new=tmp_path / "zhenxun",
)
if package_api != "jsd":
mocked_api["zhenxun_bot_plugins_metadata"].respond(404)
if package_api != "gh":
mocked_api["zhenxun_bot_plugins_tree"].respond(404)
if not is_commit:
mocked_api["zhenxun_bot_plugins_commit"].respond(404)
mocked_api["zhenxun_bot_plugins_commit_proxy"].respond(404)
mocked_api["zhenxun_bot_plugins_index_commit"].respond(404)
mocked_api["zhenxun_bot_plugins_index_commit_proxy"].respond(404)
plugin_id = 1
plugin_id = "search_image"
async with app.test_matcher(_matcher) as ctx:
bot = create_bot(ctx)
@ -65,7 +47,7 @@ async def test_add_plugin_basic(
ctx.receive_event(bot=bot, event=event)
ctx.should_call_send(
event=event,
message=Message(message=f"正在添加插件 Id: {plugin_id}"),
message=Message(message=f"正在添加插件 Module: {plugin_id}"),
result=None,
bot=bot,
)
@ -75,25 +57,12 @@ async def test_add_plugin_basic(
result=None,
bot=bot,
)
if is_commit:
assert mocked_api["search_image_plugin_file_init_commit"].called
assert mocked_api["basic_plugins"].called
assert mocked_api["extra_plugins"].called
else:
assert mocked_api["search_image_plugin_file_init"].called
assert mocked_api["basic_plugins_no_commit"].called
assert mocked_api["extra_plugins_no_commit"].called
assert (mock_base_path / "plugins" / "search_image" / "__init__.py").is_file()
@pytest.mark.parametrize("package_api", ["jsd", "gh"])
@pytest.mark.parametrize("is_commit", [True, False])
async def test_add_plugin_basic_commit_version(
package_api: str,
is_commit: bool,
app: App,
mocker: MockerFixture,
mocked_api: MockRouter,
create_bot: Callable,
tmp_path: Path,
) -> None:
@ -102,23 +71,12 @@ async def test_add_plugin_basic_commit_version(
"""
from zhenxun.builtin_plugins.plugin_store import _matcher
init_mocked_api(mocked_api=mocked_api)
mock_base_path = mocker.patch(
"zhenxun.builtin_plugins.plugin_store.data_source.BASE_PATH",
new=tmp_path / "zhenxun",
)
if package_api != "jsd":
mocked_api["zhenxun_bot_plugins_metadata_commit"].respond(404)
if package_api != "gh":
mocked_api["zhenxun_bot_plugins_tree_commit"].respond(404)
if not is_commit:
mocked_api["zhenxun_bot_plugins_commit"].respond(404)
mocked_api["zhenxun_bot_plugins_commit_proxy"].respond(404)
mocked_api["zhenxun_bot_plugins_index_commit"].respond(404)
mocked_api["zhenxun_bot_plugins_index_commit_proxy"].respond(404)
plugin_id = 3
plugin_id = "bilibili_sub"
async with app.test_matcher(_matcher) as ctx:
bot = create_bot(ctx)
@ -135,7 +93,7 @@ async def test_add_plugin_basic_commit_version(
ctx.receive_event(bot=bot, event=event)
ctx.should_call_send(
event=event,
message=Message(message=f"正在添加插件 Id: {plugin_id}"),
message=Message(message=f"正在添加插件 Module: {plugin_id}"),
result=None,
bot=bot,
)
@ -145,28 +103,12 @@ async def test_add_plugin_basic_commit_version(
result=None,
bot=bot,
)
if package_api == "jsd":
assert mocked_api["zhenxun_bot_plugins_metadata_commit"].called
if package_api == "gh":
assert mocked_api["zhenxun_bot_plugins_tree_commit"].called
if is_commit:
assert mocked_api["basic_plugins"].called
assert mocked_api["extra_plugins"].called
else:
assert mocked_api["basic_plugins_no_commit"].called
assert mocked_api["extra_plugins_no_commit"].called
assert mocked_api["bilibili_sub_plugin_file_init"].called
assert (mock_base_path / "plugins" / "bilibili_sub" / "__init__.py").is_file()
@pytest.mark.parametrize("package_api", ["jsd", "gh"])
@pytest.mark.parametrize("is_commit", [True, False])
async def test_add_plugin_basic_is_not_dir(
package_api: str,
is_commit: bool,
app: App,
mocker: MockerFixture,
mocked_api: MockRouter,
create_bot: Callable,
tmp_path: Path,
) -> None:
@ -175,24 +117,12 @@ async def test_add_plugin_basic_is_not_dir(
"""
from zhenxun.builtin_plugins.plugin_store import _matcher
init_mocked_api(mocked_api=mocked_api)
mock_base_path = mocker.patch(
"zhenxun.builtin_plugins.plugin_store.data_source.BASE_PATH",
new=tmp_path / "zhenxun",
)
if package_api != "jsd":
mocked_api["zhenxun_bot_plugins_metadata"].respond(404)
if package_api != "gh":
mocked_api["zhenxun_bot_plugins_tree"].respond(404)
if not is_commit:
mocked_api["zhenxun_bot_plugins_commit"].respond(404)
mocked_api["zhenxun_bot_plugins_commit_proxy"].respond(404)
mocked_api["zhenxun_bot_plugins_index_commit"].respond(404)
mocked_api["zhenxun_bot_plugins_index_commit_proxy"].respond(404)
plugin_id = 0
plugin_id = "jitang"
async with app.test_matcher(_matcher) as ctx:
bot = create_bot(ctx)
@ -209,7 +139,7 @@ async def test_add_plugin_basic_is_not_dir(
ctx.receive_event(bot=bot, event=event)
ctx.should_call_send(
event=event,
message=Message(message=f"正在添加插件 Id: {plugin_id}"),
message=Message(message=f"正在添加插件 Module: {plugin_id}"),
result=None,
bot=bot,
)
@ -219,25 +149,12 @@ async def test_add_plugin_basic_is_not_dir(
result=None,
bot=bot,
)
if is_commit:
assert mocked_api["jitang_plugin_file_commit"].called
assert mocked_api["basic_plugins"].called
assert mocked_api["extra_plugins"].called
else:
assert mocked_api["jitang_plugin_file"].called
assert mocked_api["basic_plugins_no_commit"].called
assert mocked_api["extra_plugins_no_commit"].called
assert (mock_base_path / "plugins" / "alapi" / "jitang.py").is_file()
assert (mock_base_path / "plugins" / "jitang.py").is_file()
@pytest.mark.parametrize("package_api", ["jsd", "gh"])
@pytest.mark.parametrize("is_commit", [True, False])
async def test_add_plugin_extra(
package_api: str,
is_commit: bool,
app: App,
mocker: MockerFixture,
mocked_api: MockRouter,
create_bot: Callable,
tmp_path: Path,
) -> None:
@ -246,26 +163,12 @@ async def test_add_plugin_extra(
"""
from zhenxun.builtin_plugins.plugin_store import _matcher
init_mocked_api(mocked_api=mocked_api)
mock_base_path = mocker.patch(
"zhenxun.builtin_plugins.plugin_store.data_source.BASE_PATH",
new=tmp_path / "zhenxun",
)
if package_api != "jsd":
mocked_api["zhenxun_github_sub_metadata"].respond(404)
if package_api != "gh":
mocked_api["zhenxun_github_sub_tree"].respond(404)
if not is_commit:
mocked_api["zhenxun_github_sub_commit"].respond(404)
mocked_api["zhenxun_github_sub_commit_proxy"].respond(404)
mocked_api["zhenxun_bot_plugins_commit"].respond(404)
mocked_api["zhenxun_bot_plugins_commit_proxy"].respond(404)
mocked_api["zhenxun_bot_plugins_index_commit"].respond(404)
mocked_api["zhenxun_bot_plugins_index_commit_proxy"].respond(404)
plugin_id = 4
plugin_id = "github_sub"
async with app.test_matcher(_matcher) as ctx:
bot = create_bot(ctx)
@ -282,7 +185,7 @@ async def test_add_plugin_extra(
ctx.receive_event(bot=bot, event=event)
ctx.should_call_send(
event=event,
message=Message(message=f"正在添加插件 Id: {plugin_id}"),
message=Message(message=f"正在添加插件 Module: {plugin_id}"),
result=None,
bot=bot,
)
@ -292,30 +195,18 @@ async def test_add_plugin_extra(
result=None,
bot=bot,
)
if is_commit:
assert mocked_api["github_sub_plugin_file_init_commit"].called
assert mocked_api["basic_plugins"].called
assert mocked_api["extra_plugins"].called
else:
assert mocked_api["github_sub_plugin_file_init"].called
assert mocked_api["basic_plugins_no_commit"].called
assert mocked_api["extra_plugins_no_commit"].called
assert (mock_base_path / "plugins" / "github_sub" / "__init__.py").is_file()
async def test_plugin_not_exist_add(
app: App,
mocker: MockerFixture,
mocked_api: MockRouter,
create_bot: Callable,
tmp_path: Path,
) -> None:
"""
测试插件不存在，添加插件
"""
from zhenxun.builtin_plugins.plugin_store import _matcher
init_mocked_api(mocked_api=mocked_api)
plugin_id = -1
async with app.test_matcher(_matcher) as ctx:
@ -339,7 +230,7 @@ async def test_plugin_not_exist_add(
)
ctx.should_call_send(
event=event,
message=Message(message="插件ID不存在..."),
message=Message(message="添加插件 Id: -1 失败 e: 插件ID不存在..."),
result=None,
bot=bot,
)
@ -348,16 +239,13 @@ async def test_plugin_not_exist_add(
async def test_add_plugin_exist(
app: App,
mocker: MockerFixture,
mocked_api: MockRouter,
create_bot: Callable,
tmp_path: Path,
) -> None:
"""
测试插件已经存在，添加插件
"""
from zhenxun.builtin_plugins.plugin_store import _matcher
init_mocked_api(mocked_api=mocked_api)
mocker.patch(
"zhenxun.builtin_plugins.plugin_store.data_source.StoreManager.get_loaded_plugins",
return_value=[("search_image", "0.1")],
@ -385,7 +273,9 @@ async def test_add_plugin_exist(
)
ctx.should_call_send(
event=event,
message=Message(message="插件 识图 已安装，无需重复安装"),
message=Message(
message="添加插件 Id: 1 失败 e: 插件 识图 已安装，无需重复安装"
),
result=None,
bot=bot,
)

140
tests/builtin_plugins/plugin_store/test_plugin_store.py
View File

@ -1,140 +0,0 @@
from collections.abc import Callable
from pathlib import Path
from typing import cast
from nonebot.adapters.onebot.v11 import Bot, Message
from nonebot.adapters.onebot.v11.event import GroupMessageEvent
from nonebug import App
from pytest_mock import MockerFixture
from respx import MockRouter
from tests.builtin_plugins.plugin_store.utils import init_mocked_api
from tests.config import BotId, GroupId, MessageId, UserId
from tests.utils import _v11_group_message_event
async def test_plugin_store(
app: App,
mocker: MockerFixture,
mocked_api: MockRouter,
create_bot: Callable,
tmp_path: Path,
) -> None:
"""
测试插件商店
"""
from zhenxun.builtin_plugins.plugin_store import _matcher
from zhenxun.builtin_plugins.plugin_store.data_source import row_style
init_mocked_api(mocked_api=mocked_api)
mock_table_page = mocker.patch(
"zhenxun.builtin_plugins.plugin_store.data_source.ImageTemplate.table_page"
)
mock_table_page_return = mocker.AsyncMock()
mock_table_page.return_value = mock_table_page_return
mock_build_message = mocker.patch(
"zhenxun.builtin_plugins.plugin_store.MessageUtils.build_message"
)
mock_build_message_return = mocker.AsyncMock()
mock_build_message.return_value = mock_build_message_return
async with app.test_matcher(_matcher) as ctx:
bot = create_bot(ctx)
bot: Bot = cast(Bot, bot)
raw_message = "插件商店"
event: GroupMessageEvent = _v11_group_message_event(
message=raw_message,
self_id=BotId.QQ_BOT,
user_id=UserId.SUPERUSER,
group_id=GroupId.GROUP_ID_LEVEL_5,
message_id=MessageId.MESSAGE_ID_3,
to_me=True,
)
ctx.receive_event(bot=bot, event=event)
mock_table_page.assert_awaited_once_with(
"插件列表",
"通过添加/移除插件 ID 来管理插件",
["-", "ID", "名称", "简介", "作者", "版本", "类型"],
[
["", 0, "鸡汤", "喏，亲手为你煮的鸡汤", "HibiKier", "0.1", "普通插件"],
["", 1, "识图", "以图搜图，看破本源", "HibiKier", "0.1", "普通插件"],
["", 2, "网易云热评", "生了个人，我很抱歉", "HibiKier", "0.1", "普通插件"],
[
"",
3,
"B站订阅",
"非常便利的B站订阅通知",
"HibiKier",
"0.3-b101fbc",
"普通插件",
],
[
"",
4,
"github订阅",
"订阅github用户或仓库",
"xuanerwa",
"0.7",
"普通插件",
],
[
"",
5,
"Minecraft查服",
"Minecraft服务器状态查询，支持IPv6",
"molanp",
"1.13",
"普通插件",
],
],
text_style=row_style,
)
mock_build_message.assert_called_once_with(mock_table_page_return)
mock_build_message_return.send.assert_awaited_once()
assert mocked_api["basic_plugins"].called
assert mocked_api["extra_plugins"].called
async def test_plugin_store_fail(
app: App,
mocker: MockerFixture,
mocked_api: MockRouter,
create_bot: Callable,
tmp_path: Path,
) -> None:
"""
测试插件商店
"""
from zhenxun.builtin_plugins.plugin_store import _matcher
init_mocked_api(mocked_api=mocked_api)
mocked_api.get(
"https://raw.githubusercontent.com/zhenxun-org/zhenxun_bot_plugins/b101fbc/plugins.json",
name="basic_plugins",
).respond(404)
async with app.test_matcher(_matcher) as ctx:
bot = create_bot(ctx)
bot: Bot = cast(Bot, bot)
raw_message = "插件商店"
event: GroupMessageEvent = _v11_group_message_event(
message=raw_message,
self_id=BotId.QQ_BOT,
user_id=UserId.SUPERUSER,
group_id=GroupId.GROUP_ID_LEVEL_5,
message_id=MessageId.MESSAGE_ID_3,
to_me=True,
)
ctx.receive_event(bot=bot, event=event)
ctx.should_call_send(
event=event,
message=Message("获取插件列表失败..."),
result=None,
exception=None,
bot=bot,
)
assert mocked_api["basic_plugins"].called

16
tests/builtin_plugins/plugin_store/test_remove_plugin.py
View File

@ -9,9 +9,7 @@ from nonebot.adapters.onebot.v11.event import GroupMessageEvent
from nonebot.adapters.onebot.v11.message import Message
from nonebug import App
from pytest_mock import MockerFixture
from respx import MockRouter
from tests.builtin_plugins.plugin_store.utils import get_content_bytes, init_mocked_api
from tests.config import BotId, GroupId, MessageId, UserId
from tests.utils import _v11_group_message_event
@ -19,7 +17,6 @@ from tests.utils import _v11_group_message_event
async def test_remove_plugin(
app: App,
mocker: MockerFixture,
mocked_api: MockRouter,
create_bot: Callable,
tmp_path: Path,
) -> None:
@ -28,7 +25,6 @@ async def test_remove_plugin(
"""
from zhenxun.builtin_plugins.plugin_store import _matcher
init_mocked_api(mocked_api=mocked_api)
mock_base_path = mocker.patch(
"zhenxun.builtin_plugins.plugin_store.data_source.BASE_PATH",
new=tmp_path / "zhenxun",
@ -38,7 +34,7 @@ async def test_remove_plugin(
plugin_path.mkdir(parents=True, exist_ok=True)
with open(plugin_path / "__init__.py", "wb") as f:
f.write(get_content_bytes("search_image.py"))
f.write(b"A_nmi")
plugin_id = 1
@ -61,24 +57,18 @@ async def test_remove_plugin(
result=None,
bot=bot,
)
assert mocked_api["basic_plugins"].called
assert mocked_api["extra_plugins"].called
assert not (mock_base_path / "plugins" / "search_image" / "__init__.py").is_file()
async def test_plugin_not_exist_remove(
app: App,
mocker: MockerFixture,
mocked_api: MockRouter,
create_bot: Callable,
tmp_path: Path,
) -> None:
"""
测试插件不存在，移除插件
"""
from zhenxun.builtin_plugins.plugin_store import _matcher
init_mocked_api(mocked_api=mocked_api)
plugin_id = -1
async with app.test_matcher(_matcher) as ctx:
@ -96,7 +86,7 @@ async def test_plugin_not_exist_remove(
ctx.receive_event(bot=bot, event=event)
ctx.should_call_send(
event=event,
message=Message(message="插件ID不存在..."),
message=Message(message="移除插件 Id: -1 失败 e: 插件ID不存在..."),
result=None,
bot=bot,
)
@ -105,7 +95,6 @@ async def test_plugin_not_exist_remove(
async def test_remove_plugin_not_install(
app: App,
mocker: MockerFixture,
mocked_api: MockRouter,
create_bot: Callable,
tmp_path: Path,
) -> None:
@ -114,7 +103,6 @@ async def test_remove_plugin_not_install(
"""
from zhenxun.builtin_plugins.plugin_store import _matcher
init_mocked_api(mocked_api=mocked_api)
_ = mocker.patch(
"zhenxun.builtin_plugins.plugin_store.data_source.BASE_PATH",
new=tmp_path / "zhenxun",

57
tests/builtin_plugins/plugin_store/test_search_plugin.py
View File

@ -1,5 +1,4 @@
from collections.abc import Callable
from pathlib import Path
from typing import cast
from nonebot.adapters.onebot.v11 import Bot
@ -7,9 +6,7 @@ from nonebot.adapters.onebot.v11.event import GroupMessageEvent
from nonebot.adapters.onebot.v11.message import Message
from nonebug import App
from pytest_mock import MockerFixture
from respx import MockRouter
from tests.builtin_plugins.plugin_store.utils import init_mocked_api
from tests.config import BotId, GroupId, MessageId, UserId
from tests.utils import _v11_group_message_event
@ -17,17 +14,12 @@ from tests.utils import _v11_group_message_event
async def test_search_plugin_name(
app: App,
mocker: MockerFixture,
mocked_api: MockRouter,
create_bot: Callable,
tmp_path: Path,
) -> None:
"""
测试搜索插件
"""
from zhenxun.builtin_plugins.plugin_store import _matcher
from zhenxun.builtin_plugins.plugin_store.data_source import row_style
init_mocked_api(mocked_api=mocked_api)
mock_table_page = mocker.patch(
"zhenxun.builtin_plugins.plugin_store.data_source.ImageTemplate.table_page"
@ -56,44 +48,19 @@ async def test_search_plugin_name(
to_me=True,
)
ctx.receive_event(bot=bot, event=event)
mock_table_page.assert_awaited_once_with(
"商店插件列表",
"通过添加/移除插件 ID 来管理插件",
["-", "ID", "名称", "简介", "作者", "版本", "类型"],
[
[
"",
4,
"github订阅",
"订阅github用户或仓库",
"xuanerwa",
"0.7",
"普通插件",
]
],
text_style=row_style,
)
mock_build_message.assert_called_once_with(mock_table_page_return)
mock_build_message_return.send.assert_awaited_once()
assert mocked_api["basic_plugins"].called
assert mocked_api["extra_plugins"].called
async def test_search_plugin_author(
app: App,
mocker: MockerFixture,
mocked_api: MockRouter,
create_bot: Callable,
tmp_path: Path,
) -> None:
"""
测试搜索插件，作者
"""
from zhenxun.builtin_plugins.plugin_store import _matcher
from zhenxun.builtin_plugins.plugin_store.data_source import row_style
init_mocked_api(mocked_api=mocked_api)
mock_table_page = mocker.patch(
"zhenxun.builtin_plugins.plugin_store.data_source.ImageTemplate.table_page"
@ -122,43 +89,19 @@ async def test_search_plugin_author(
to_me=True,
)
ctx.receive_event(bot=bot, event=event)
mock_table_page.assert_awaited_once_with(
"商店插件列表",
"通过添加/移除插件 ID 来管理插件",
["-", "ID", "名称", "简介", "作者", "版本", "类型"],
[
[
"",
4,
"github订阅",
"订阅github用户或仓库",
"xuanerwa",
"0.7",
"普通插件",
]
],
text_style=row_style,
)
mock_build_message.assert_called_once_with(mock_table_page_return)
mock_build_message_return.send.assert_awaited_once()
assert mocked_api["basic_plugins"].called
assert mocked_api["extra_plugins"].called
async def test_plugin_not_exist_search(
app: App,
mocker: MockerFixture,
mocked_api: MockRouter,
create_bot: Callable,
tmp_path: Path,
) -> None:
"""
测试插件不存在，搜索插件
"""
from zhenxun.builtin_plugins.plugin_store import _matcher
init_mocked_api(mocked_api=mocked_api)
plugin_name = "not_exist_plugin_name"
async with app.test_matcher(_matcher) as ctx:

13
tests/builtin_plugins/plugin_store/test_update_all_plugin.py
View File

@ -7,9 +7,7 @@ from nonebot.adapters.onebot.v11.event import GroupMessageEvent
from nonebot.adapters.onebot.v11.message import Message
from nonebug import App
from pytest_mock import MockerFixture
from respx import MockRouter
from tests.builtin_plugins.plugin_store.utils import init_mocked_api
from tests.config import BotId, GroupId, MessageId, UserId
from tests.utils import _v11_group_message_event
@ -17,7 +15,6 @@ from tests.utils import _v11_group_message_event
async def test_update_all_plugin_basic_need_update(
app: App,
mocker: MockerFixture,
mocked_api: MockRouter,
create_bot: Callable,
tmp_path: Path,
) -> None:
@ -26,7 +23,6 @@ async def test_update_all_plugin_basic_need_update(
"""
from zhenxun.builtin_plugins.plugin_store import _matcher
init_mocked_api(mocked_api=mocked_api)
mock_base_path = mocker.patch(
"zhenxun.builtin_plugins.plugin_store.data_source.BASE_PATH",
new=tmp_path / "zhenxun",
@ -63,16 +59,12 @@ async def test_update_all_plugin_basic_need_update(
result=None,
bot=bot,
)
assert mocked_api["basic_plugins"].called
assert mocked_api["extra_plugins"].called
assert mocked_api["search_image_plugin_file_init_commit"].called
assert (mock_base_path / "plugins" / "search_image" / "__init__.py").is_file()
async def test_update_all_plugin_basic_is_new(
app: App,
mocker: MockerFixture,
mocked_api: MockRouter,
create_bot: Callable,
tmp_path: Path,
) -> None:
@ -81,14 +73,13 @@ async def test_update_all_plugin_basic_is_new(
"""
from zhenxun.builtin_plugins.plugin_store import _matcher
init_mocked_api(mocked_api=mocked_api)
mocker.patch(
"zhenxun.builtin_plugins.plugin_store.data_source.BASE_PATH",
new=tmp_path / "zhenxun",
)
mocker.patch(
"zhenxun.builtin_plugins.plugin_store.data_source.StoreManager.get_loaded_plugins",
return_value=[("search_image", "0.1")],
return_value=[("search_image", "0.2")],
)
async with app.test_matcher(_matcher) as ctx:
@ -116,5 +107,3 @@ async def test_update_all_plugin_basic_is_new(
result=None,
bot=bot,
)
assert mocked_api["basic_plugins"].called
assert mocked_api["extra_plugins"].called

25
tests/builtin_plugins/plugin_store/test_update_plugin.py
View File

@ -9,7 +9,6 @@ from nonebug import App
from pytest_mock import MockerFixture
from respx import MockRouter
from tests.builtin_plugins.plugin_store.utils import init_mocked_api
from tests.config import BotId, GroupId, MessageId, UserId
from tests.utils import _v11_group_message_event
@ -17,7 +16,6 @@ from tests.utils import _v11_group_message_event
async def test_update_plugin_basic_need_update(
app: App,
mocker: MockerFixture,
mocked_api: MockRouter,
create_bot: Callable,
tmp_path: Path,
) -> None:
@ -26,7 +24,6 @@ async def test_update_plugin_basic_need_update(
"""
from zhenxun.builtin_plugins.plugin_store import _matcher
init_mocked_api(mocked_api=mocked_api)
mock_base_path = mocker.patch(
"zhenxun.builtin_plugins.plugin_store.data_source.BASE_PATH",
new=tmp_path / "zhenxun",
@ -63,16 +60,12 @@ async def test_update_plugin_basic_need_update(
result=None,
bot=bot,
)
assert mocked_api["basic_plugins"].called
assert mocked_api["extra_plugins"].called
assert mocked_api["search_image_plugin_file_init_commit"].called
assert (mock_base_path / "plugins" / "search_image" / "__init__.py").is_file()
async def test_update_plugin_basic_is_new(
app: App,
mocker: MockerFixture,
mocked_api: MockRouter,
create_bot: Callable,
tmp_path: Path,
) -> None:
@ -81,14 +74,13 @@ async def test_update_plugin_basic_is_new(
"""
from zhenxun.builtin_plugins.plugin_store import _matcher
init_mocked_api(mocked_api=mocked_api)
mocker.patch(
"zhenxun.builtin_plugins.plugin_store.data_source.BASE_PATH",
new=tmp_path / "zhenxun",
)
mocker.patch(
"zhenxun.builtin_plugins.plugin_store.data_source.StoreManager.get_loaded_plugins",
return_value=[("search_image", "0.1")],
return_value=[("search_image", "0.2")],
)
plugin_id = 1
@ -118,23 +110,17 @@ async def test_update_plugin_basic_is_new(
result=None,
bot=bot,
)
assert mocked_api["basic_plugins"].called
assert mocked_api["extra_plugins"].called
async def test_plugin_not_exist_update(
app: App,
mocker: MockerFixture,
mocked_api: MockRouter,
create_bot: Callable,
tmp_path: Path,
) -> None:
"""
测试插件不存在，更新插件
"""
from zhenxun.builtin_plugins.plugin_store import _matcher
init_mocked_api(mocked_api=mocked_api)
plugin_id = -1
async with app.test_matcher(_matcher) as ctx:
@ -158,7 +144,7 @@ async def test_plugin_not_exist_update(
)
ctx.should_call_send(
event=event,
message=Message(message="插件ID不存在..."),
message=Message(message="更新插件 Id: -1 失败 e: 插件ID不存在..."),
result=None,
bot=bot,
)
@ -166,17 +152,14 @@ async def test_plugin_not_exist_update(
async def test_update_plugin_not_install(
app: App,
mocker: MockerFixture,
mocked_api: MockRouter,
create_bot: Callable,
tmp_path: Path,
) -> None:
"""
测试插件不存在，更新插件
"""
from zhenxun.builtin_plugins.plugin_store import _matcher
init_mocked_api(mocked_api=mocked_api)
plugin_id = 1
async with app.test_matcher(_matcher) as ctx:
@ -200,7 +183,9 @@ async def test_update_plugin_not_install(
)
ctx.should_call_send(
event=event,
message=Message(message="插件 识图 未安装，无法更新"),
message=Message(
message="更新插件 Id: 1 失败 e: 插件 识图 未安装，无法更新"
),
result=None,
bot=bot,
)

147
tests/builtin_plugins/plugin_store/utils.py
View File

@ -1,147 +0,0 @@
# ruff: noqa: ASYNC230
from pathlib import Path
from respx import MockRouter
from tests.utils import get_content_bytes as _get_content_bytes
from tests.utils import get_response_json as _get_response_json
def get_response_json(file: str) -> dict:
return _get_response_json(Path() / "plugin_store", file=file)
def get_content_bytes(file: str) -> bytes:
return _get_content_bytes(Path() / "plugin_store", file)
def init_mocked_api(mocked_api: MockRouter) -> None:
# metadata
mocked_api.get(
"https://data.jsdelivr.com/v1/packages/gh/zhenxun-org/zhenxun_bot_plugins@main",
name="zhenxun_bot_plugins_metadata",
).respond(json=get_response_json("zhenxun_bot_plugins_metadata.json"))
mocked_api.get(
"https://data.jsdelivr.com/v1/packages/gh/xuanerwa/zhenxun_github_sub@main",
name="zhenxun_github_sub_metadata",
).respond(json=get_response_json("zhenxun_github_sub_metadata.json"))
mocked_api.get(
"https://data.jsdelivr.com/v1/packages/gh/zhenxun-org/zhenxun_bot_plugins@b101fbc",
name="zhenxun_bot_plugins_metadata_commit",
).respond(json=get_response_json("zhenxun_bot_plugins_metadata.json"))
mocked_api.get(
"https://data.jsdelivr.com/v1/packages/gh/xuanerwa/zhenxun_github_sub@f524632f78d27f9893beebdf709e0e7885cd08f1",
name="zhenxun_github_sub_metadata_commit",
).respond(json=get_response_json("zhenxun_github_sub_metadata.json"))
# tree
mocked_api.get(
"https://api.github.com/repos/zhenxun-org/zhenxun_bot_plugins/git/trees/main?recursive=1",
name="zhenxun_bot_plugins_tree",
).respond(json=get_response_json("zhenxun_bot_plugins_tree.json"))
mocked_api.get(
"https://api.github.com/repos/xuanerwa/zhenxun_github_sub/git/trees/main?recursive=1",
name="zhenxun_github_sub_tree",
).respond(json=get_response_json("zhenxun_github_sub_tree.json"))
mocked_api.get(
"https://api.github.com/repos/zhenxun-org/zhenxun_bot_plugins/git/trees/b101fbc?recursive=1",
name="zhenxun_bot_plugins_tree_commit",
).respond(json=get_response_json("zhenxun_bot_plugins_tree.json"))
mocked_api.get(
"https://api.github.com/repos/xuanerwa/zhenxun_github_sub/git/trees/f524632f78d27f9893beebdf709e0e7885cd08f1?recursive=1",
name="zhenxun_github_sub_tree_commit",
).respond(json=get_response_json("zhenxun_github_sub_tree.json"))
mocked_api.head(
"https://raw.githubusercontent.com/",
name="head_raw",
).respond(200, text="")
mocked_api.get(
"https://raw.githubusercontent.com/zhenxun-org/zhenxun_bot_plugins/b101fbc/plugins.json",
name="basic_plugins",
).respond(json=get_response_json("basic_plugins.json"))
mocked_api.get(
"https://cdn.jsdelivr.net/gh/zhenxun-org/zhenxun_bot_plugins@b101fbc/plugins.json",
name="basic_plugins_jsdelivr",
).respond(200, json=get_response_json("basic_plugins.json"))
mocked_api.get(
"https://raw.githubusercontent.com/zhenxun-org/zhenxun_bot_plugins/main/plugins.json",
name="basic_plugins_no_commit",
).respond(json=get_response_json("basic_plugins.json"))
mocked_api.get(
"https://cdn.jsdelivr.net/gh/zhenxun-org/zhenxun_bot_plugins@main/plugins.json",
name="basic_plugins_jsdelivr_no_commit",
).respond(200, json=get_response_json("basic_plugins.json"))
mocked_api.get(
"https://raw.githubusercontent.com/zhenxun-org/zhenxun_bot_plugins_index/2ed61284873c526802752b12a3fd3b5e1a59d948/plugins.json",
name="extra_plugins",
).respond(200, json=get_response_json("extra_plugins.json"))
mocked_api.get(
"https://cdn.jsdelivr.net/gh/zhenxun-org/zhenxun_bot_plugins_index@2ed61284873c526802752b12a3fd3b5e1a59d948/plugins.json",
name="extra_plugins_jsdelivr",
).respond(200, json=get_response_json("extra_plugins.json"))
mocked_api.get(
"https://raw.githubusercontent.com/zhenxun-org/zhenxun_bot_plugins_index/index/plugins.json",
name="extra_plugins_no_commit",
).respond(200, json=get_response_json("extra_plugins.json"))
mocked_api.get(
"https://cdn.jsdelivr.net/gh/zhenxun-org/zhenxun_bot_plugins_index@index/plugins.json",
name="extra_plugins_jsdelivr_no_commit",
).respond(200, json=get_response_json("extra_plugins.json"))
mocked_api.get(
"https://raw.githubusercontent.com/zhenxun-org/zhenxun_bot_plugins/main/plugins/search_image/__init__.py",
name="search_image_plugin_file_init",
).respond(content=get_content_bytes("search_image.py"))
mocked_api.get(
"https://raw.githubusercontent.com/zhenxun-org/zhenxun_bot_plugins/b101fbc/plugins/search_image/__init__.py",
name="search_image_plugin_file_init_commit",
).respond(content=get_content_bytes("search_image.py"))
mocked_api.get(
"https://raw.githubusercontent.com/zhenxun-org/zhenxun_bot_plugins/main/plugins/alapi/jitang.py",
name="jitang_plugin_file",
).respond(content=get_content_bytes("jitang.py"))
mocked_api.get(
"https://raw.githubusercontent.com/zhenxun-org/zhenxun_bot_plugins/b101fbc/plugins/alapi/jitang.py",
name="jitang_plugin_file_commit",
).respond(content=get_content_bytes("jitang.py"))
mocked_api.get(
"https://raw.githubusercontent.com/xuanerwa/zhenxun_github_sub/main/github_sub/__init__.py",
name="github_sub_plugin_file_init",
).respond(content=get_content_bytes("github_sub.py"))
mocked_api.get(
"https://raw.githubusercontent.com/xuanerwa/zhenxun_github_sub/f524632f78d27f9893beebdf709e0e7885cd08f1/github_sub/__init__.py",
name="github_sub_plugin_file_init_commit",
).respond(content=get_content_bytes("github_sub.py"))
mocked_api.get(
"https://raw.githubusercontent.com/zhenxun-org/zhenxun_bot_plugins/b101fbc/plugins/bilibili_sub/__init__.py",
name="bilibili_sub_plugin_file_init",
).respond(content=get_content_bytes("bilibili_sub.py"))
mocked_api.get(
"https://api.github.com/repos/zhenxun-org/zhenxun_bot_plugins/commits/main",
name="zhenxun_bot_plugins_commit",
).respond(json=get_response_json("zhenxun_bot_plugins_commit.json"))
mocked_api.get(
"https://git-api.zhenxun.org/repos/zhenxun-org/zhenxun_bot_plugins/commits/main",
name="zhenxun_bot_plugins_commit_proxy",
).respond(json=get_response_json("zhenxun_bot_plugins_commit.json"))
mocked_api.get(
"https://api.github.com/repos/zhenxun-org/zhenxun_bot_plugins_index/commits/index",
name="zhenxun_bot_plugins_index_commit",
).respond(json=get_response_json("zhenxun_bot_plugins_index_commit.json"))
mocked_api.get(
"https://git-api.zhenxun.org/repos/zhenxun-org/zhenxun_bot_plugins_index/commits/index",
name="zhenxun_bot_plugins_index_commit_proxy",
).respond(json=get_response_json("zhenxun_bot_plugins_index_commit.json"))
mocked_api.get(
"https://api.github.com/repos/xuanerwa/zhenxun_github_sub/commits/main",
name="zhenxun_github_sub_commit",
).respond(json=get_response_json("zhenxun_github_sub_commit.json"))
mocked_api.get(
"https://git-api.zhenxun.org/repos/xuanerwa/zhenxun_github_sub/commits/main",
name="zhenxun_github_sub_commit_proxy",
).respond(json=get_response_json("zhenxun_github_sub_commit.json"))

37
tests/content/plugin_store/bilibili_sub.py
View File

@ -1,37 +0,0 @@
from nonebot.plugin import PluginMetadata
from zhenxun.configs.utils import PluginExtraData
__plugin_meta__ = PluginMetadata(
name="B站订阅",
description="非常便利的B站订阅通知",
usage="""
usage：
B站直播，番剧，UP动态开播等提醒
主播订阅相当于 直播间订阅 + UP订阅
指令：
添加订阅 ['主播'/'UP'/'番剧'] [id/链接/番名]
删除订阅 ['主播'/'UP'/'id'] [id]
查看订阅
示例：
添加订阅主播 2345344 <-(直播房间id)
添加订阅UP 2355543 <-(个人主页id)
添加订阅番剧 史莱姆 <-(支持模糊搜索)
添加订阅番剧 125344 <-(番剧id)
删除订阅id 2324344 <-(任意id，通过查看订阅获取)
""".strip(),
extra=PluginExtraData(
author="HibiKier",
version="0.3-b101fbc",
superuser_help="""
登录b站获取cookie防止风控：
bil_check/检测b站
bil_login/登录b站
bil_logout/退出b站 uid
示例:
登录b站
检测b站
bil_logout 12345<-(退出登录的b站uid，通过检测b站获取)
""",
).to_dict(),
)

24
tests/content/plugin_store/github_sub.py
View File

@ -1,24 +0,0 @@
from nonebot.plugin import PluginMetadata
from zhenxun.configs.utils import PluginExtraData
__plugin_meta__ = PluginMetadata(
name="github订阅",
description="订阅github用户或仓库",
usage="""
usage：
github新Comment，PR，Issue等提醒
指令：
添加github ['用户'/'仓库'] [用户名/{owner/repo}]
删除github [用户名/{owner/repo}]
查看github
示例：添加github订阅 用户 HibiKier
示例：添加gb订阅 仓库 HibiKier/zhenxun_bot
示例：添加github 用户 HibiKier
示例：删除gb订阅 HibiKier
""".strip(),
extra=PluginExtraData(
author="xuanerwa",
version="0.7",
).to_dict(),
)

17
tests/content/plugin_store/jitang.py
View File

@ -1,17 +0,0 @@
from nonebot.plugin import PluginMetadata
from zhenxun.configs.utils import PluginExtraData
__plugin_meta__ = PluginMetadata(
name="鸡汤",
description="喏，亲手为你煮的鸡汤",
usage="""
不喝点什么感觉有点不舒服
指令：
鸡汤
""".strip(),
extra=PluginExtraData(
author="HibiKier",
version="0.1",
).to_dict(),
)

18
tests/content/plugin_store/search_image.py
View File

@ -1,18 +0,0 @@
from nonebot.plugin import PluginMetadata
from zhenxun.configs.utils import PluginExtraData
__plugin_meta__ = PluginMetadata(
name="识图",
description="以图搜图，看破本源",
usage="""
识别图片 [二次元图片]
指令：
识图 [图片]
""".strip(),
extra=PluginExtraData(
author="HibiKier",
version="0.1",
menu_type="一些工具",
).to_dict(),
)

46
tests/response/plugin_store/basic_plugins.json
View File

@ -1,46 +0,0 @@
[
{
"name": "鸡汤",
"module": "jitang",
"module_path": "plugins.alapi.jitang",
"description": "喏，亲手为你煮的鸡汤",
"usage": "不喝点什么感觉有点不舒服\n 指令：\n 鸡汤",
"author": "HibiKier",
"version": "0.1",
"plugin_type": "NORMAL",
"is_dir": false
},
{
"name": "识图",
"module": "search_image",
"module_path": "plugins.search_image",
"description": "以图搜图，看破本源",
"usage": "识别图片 [二次元图片]\n 指令：\n 识图 [图片]",
"author": "HibiKier",
"version": "0.1",
"plugin_type": "NORMAL",
"is_dir": true
},
{
"name": "网易云热评",
"module": "comments_163",
"module_path": "plugins.alapi.comments_163",
"description": "生了个人，我很抱歉",
"usage": "到点了，还是防不了下塔\n 指令：\n 网易云热评/到点了/12点了",
"author": "HibiKier",
"version": "0.1",
"plugin_type": "NORMAL",
"is_dir": false
},
{
"name": "B站订阅",
"module": "bilibili_sub",
"module_path": "plugins.bilibili_sub",
"description": "非常便利的B站订阅通知",
"usage": "B站直播，番剧，UP动态开播等提醒",
"author": "HibiKier",
"version": "0.3-b101fbc",
"plugin_type": "NORMAL",
"is_dir": true
}
]

26
tests/response/plugin_store/extra_plugins.json
View File

@ -1,26 +0,0 @@
[
{
"name": "github订阅",
"module": "github_sub",
"module_path": "github_sub",
"description": "订阅github用户或仓库",
"usage": "usage：\n github新Comment，PR，Issue等提醒\n 指令：\n 添加github ['用户'/'仓库'] [用户名/{owner/repo}]\n 删除github [用户名/{owner/repo}]\n 查看github\n 示例：添加github订阅 用户 HibiKier\n 示例：添加gb订阅 仓库 HibiKier/zhenxun_bot\n 示例：添加github 用户 HibiKier\n 示例：删除gb订阅 HibiKier",
"author": "xuanerwa",
"version": "0.7",
"plugin_type": "NORMAL",
"is_dir": true,
"github_url": "https://github.com/xuanerwa/zhenxun_github_sub"
},
{
"name": "Minecraft查服",
"module": "mc_check",
"module_path": "mc_check",
"description": "Minecraft服务器状态查询，支持IPv6",
"usage": "Minecraft服务器状态查询，支持IPv6\n用法：\n\t查服 [ip]:[端口] / 查服 [ip]\n\t设置语言 zh-cn\n\t当前语言\n\t语言列表\neg:\t\nmcheck ip:port / mcheck ip\n\tset_lang en\n\tlang_now\n\tlang_list",
"author": "molanp",
"version": "1.13",
"plugin_type": "NORMAL",
"is_dir": true,
"github_url": "https://github.com/molanp/zhenxun_check_Minecraft"
}
]

101
tests/response/plugin_store/zhenxun_bot_plugins_commit.json
View File

@ -1,101 +0,0 @@
{
"sha": "b101fbc",
"node_id": "C_kwDOMndPGNoAKGIxMDFmYmNlODg4NjA4ZTJiYmU1YjVmZDI3OWUxNDY1MTY4ODEyYzc",
"commit": {
"author": {
"name": "xuaner",
"email": "xuaner_wa@qq.com",
"date": "2024-09-20T12:08:27Z"
},
"committer": {
"name": "xuaner",
"email": "xuaner_wa@qq.com",
"date": "2024-09-20T12:08:27Z"
},
"message": "🐛修复B站订阅bug",
"tree": {
"sha": "0566306219a434f7122798647498faef692c1879",
"url": "https://api.github.com/repos/zhenxun-org/zhenxun_bot_plugins/git/trees/0566306219a434f7122798647498faef692c1879"
},
"url": "https://api.github.com/repos/zhenxun-org/zhenxun_bot_plugins/git/commits/b101fbce888608e2bbe5b5fd279e1465168812c7",
"comment_count": 0,
"verification": {
"verified": false,
"reason": "unsigned",
"signature": null,
"payload": null,
"verified_at": null
}
},
"url": "https://api.github.com/repos/zhenxun-org/zhenxun_bot_plugins/commits/b101fbce888608e2bbe5b5fd279e1465168812c7",
"html_url": "https://github.com/zhenxun-org/zhenxun_bot_plugins/commit/b101fbce888608e2bbe5b5fd279e1465168812c7",
"comments_url": "https://api.github.com/repos/zhenxun-org/zhenxun_bot_plugins/commits/b101fbce888608e2bbe5b5fd279e1465168812c7/comments",
"author": {
"login": "xuanerwa",
"id": 58063798,
"node_id": "MDQ6VXNlcjU4MDYzNzk4",
"avatar_url": "https://avatars.githubusercontent.com/u/58063798?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/xuanerwa",
"html_url": "https://github.com/xuanerwa",
"followers_url": "https://api.github.com/users/xuanerwa/followers",
"following_url": "https://api.github.com/users/xuanerwa/following{/other_user}",
"gists_url": "https://api.github.com/users/xuanerwa/gists{/gist_id}",
"starred_url": "https://api.github.com/users/xuanerwa/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/xuanerwa/subscriptions",
"organizations_url": "https://api.github.com/users/xuanerwa/orgs",
"repos_url": "https://api.github.com/users/xuanerwa/repos",
"events_url": "https://api.github.com/users/xuanerwa/events{/privacy}",
"received_events_url": "https://api.github.com/users/xuanerwa/received_events",
"type": "User",
"user_view_type": "public",
"site_admin": false
},
"committer": {
"login": "xuanerwa",
"id": 58063798,
"node_id": "MDQ6VXNlcjU4MDYzNzk4",
"avatar_url": "https://avatars.githubusercontent.com/u/58063798?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/xuanerwa",
"html_url": "https://github.com/xuanerwa",
"followers_url": "https://api.github.com/users/xuanerwa/followers",
"following_url": "https://api.github.com/users/xuanerwa/following{/other_user}",
"gists_url": "https://api.github.com/users/xuanerwa/gists{/gist_id}",
"starred_url": "https://api.github.com/users/xuanerwa/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/xuanerwa/subscriptions",
"organizations_url": "https://api.github.com/users/xuanerwa/orgs",
"repos_url": "https://api.github.com/users/xuanerwa/repos",
"events_url": "https://api.github.com/users/xuanerwa/events{/privacy}",
"received_events_url": "https://api.github.com/users/xuanerwa/received_events",
"type": "User",
"user_view_type": "public",
"site_admin": false
},
"parents": [
{
"sha": "a545dfa0c4e149595f7ddd50dc34c55513738fb9",
"url": "https://api.github.com/repos/zhenxun-org/zhenxun_bot_plugins/commits/a545dfa0c4e149595f7ddd50dc34c55513738fb9",
"html_url": "https://github.com/zhenxun-org/zhenxun_bot_plugins/commit/a545dfa0c4e149595f7ddd50dc34c55513738fb9"
}
],
"stats": {
"total": 4,
"additions": 2,
"deletions": 2
},
"files": [
{
"sha": "0fbc9695db04c56174e3bff933f670d8d2df2abc",
"filename": "plugins/bilibili_sub/data_source.py",
"status": "modified",
"additions": 2,
"deletions": 2,
"changes": 4,
"blob_url": "https://github.com/zhenxun-org/zhenxun_bot_plugins/blob/b101fbce888608e2bbe5b5fd279e1465168812c7/plugins%2Fbilibili_sub%2Fdata_source.py",
"raw_url": "https://github.com/zhenxun-org/zhenxun_bot_plugins/raw/b101fbce888608e2bbe5b5fd279e1465168812c7/plugins%2Fbilibili_sub%2Fdata_source.py",
"contents_url": "https://api.github.com/repos/zhenxun-org/zhenxun_bot_plugins/contents/plugins%2Fbilibili_sub%2Fdata_source.py?ref=b101fbce888608e2bbe5b5fd279e1465168812c7",
"patch": "@@ -271,14 +271,14 @@ async def _get_live_status(id_: int) -> list:\n sub = await BilibiliSub.get_or_none(sub_id=id_)\n msg_list = []\n if sub.live_status != live_status:\n+ await BilibiliSub.sub_handle(id_, live_status=live_status)\n image = None\n try:\n image_bytes = await fetch_image_bytes(cover)\n image = BuildImage(background = image_bytes)\n except Exception as e:\n logger.error(f\"图片构造失败，错误信息：{e}\")\n if sub.live_status in [0, 2] and live_status == 1 and image:\n- await BilibiliSub.sub_handle(id_, live_status=live_status)\n msg_list = [\n image,\n \"\\n\",\n@@ -322,7 +322,7 @@ async def _get_up_status(id_: int) -> list:\n video = video_info[\"list\"][\"vlist\"][0]\n latest_video_created = video[\"created\"]\n msg_list = []\n- if dynamic_img:\n+ if dynamic_img and _user.dynamic_upload_time < dynamic_upload_time:\n await BilibiliSub.sub_handle(id_, dynamic_upload_time=dynamic_upload_time)\n msg_list = [f\"{uname} 发布了动态！📢\\n\", dynamic_img, f\"\\n查看详情：{link}\"]\n if ("
}
]
}

101
tests/response/plugin_store/zhenxun_bot_plugins_index_commit.json
View File

@ -1,101 +0,0 @@
{
"sha": "2ed61284873c526802752b12a3fd3b5e1a59d948",
"node_id": "C_kwDOGK5Du9oAKDJlZDYxMjg0ODczYzUyNjgwMjc1MmIxMmEzZmQzYjVlMWE1OWQ5NDg",
"commit": {
"author": {
"name": "zhenxunflow[bot]",
"email": "179375394+zhenxunflow[bot]@users.noreply.github.com",
"date": "2025-01-26T09:04:55Z"
},
"committer": {
"name": "GitHub",
"email": "noreply@github.com",
"date": "2025-01-26T09:04:55Z"
},
"message": ":beers: publish plugin AI全家桶 (#235) (#236)\n\nCo-authored-by: molanp <molanp@users.noreply.github.com>",
"tree": {
"sha": "64ea463e084b6ab0def0322c6ad53799054ec9b3",
"url": "https://api.github.com/repos/zhenxun-org/zhenxun_bot_plugins_index/git/trees/64ea463e084b6ab0def0322c6ad53799054ec9b3"
},
"url": "https://api.github.com/repos/zhenxun-org/zhenxun_bot_plugins_index/git/commits/2ed61284873c526802752b12a3fd3b5e1a59d948",
"comment_count": 0,
"verification": {
"verified": true,
"reason": "valid",
"signature": "-----BEGIN PGP SIGNATURE-----\n\nwsFcBAABCAAQBQJnlfq3CRC1aQ7uu5UhlAAA+n0QADPVjQQIHFlNcTEgdq3LGQ1X\nm8+H5N07E5JD+83LdyU9/YOvqY/WURwFsQ0T4+23icUWEOD4LB5qZIdVJBYHseto\nbJNmYd1kZxpvsONoiK/2Uk6JoeVnEQIR+dTbB0wBlbL0lRt1WtTXHpLQbFXuXn3q\nJh4SdSj283UZ6D2sBADblPZ7DqaTmLlpgwrTPx0OH5wIhcuORkzOl6x0DabcVAYu\nu5zHSKM9c7g+jEmrqRuVy+ZlZMDPN4S3gDNzEhoTn4tn+KNzSIja4n7ZMRD+1a5X\nMIP3aXcVBqCyuYc6DU76IvjlaL/MjnlPwfOtx1zu+pNxZKNaSpojtqopp3blfk0E\n8s8lD9utDgUaUrdPWgpiMDjj+oNMye91CGomNDfv0fNGUlBGT6r48qaq1z8BwAAR\nzgDsF13kDuKTTkT/6T8CdgCpJtwvxMptUr2XFRtn4xwf/gJdqrbEc4fHTOSHqxzh\ncDfXuP+Sorla4oJ0duygTsulpr/zguX8RJWJml35VjERw54ARAVvhZn19G9qQVJo\n2QIp+xtyTjkM3yTeN4UDXFt4lDuxz3+l1MBduj+CHn+WTgxyJUpX2TA1GVfni9xT\npOMOtzuDQfDIxTNB6hFjSWATb1/E5ys1lfK09n+dRhmvC/Be+b5M4WlyX3cqy/za\ns0XxuZ+CHzLfHaPxFUem\n=VYpl\n-----END PGP SIGNATURE-----\n",
"payload": "tree 64ea463e084b6ab0def0322c6ad53799054ec9b3\nparent 5df26081d40e3000a7beedb73954d4df397c93fa\nauthor zhenxunflow[bot] <179375394+zhenxunflow[bot]@users.noreply.github.com> 1737882295 +0800\ncommitter GitHub <noreply@github.com> 1737882295 +0800\n\n:beers: publish plugin AI全家桶 (#235) (#236)\n\nCo-authored-by: molanp <molanp@users.noreply.github.com>",
"verified_at": "2025-01-26T09:04:58Z"
}
},
"url": "https://api.github.com/repos/zhenxun-org/zhenxun_bot_plugins_index/commits/2ed61284873c526802752b12a3fd3b5e1a59d948",
"html_url": "https://github.com/zhenxun-org/zhenxun_bot_plugins_index/commit/2ed61284873c526802752b12a3fd3b5e1a59d948",
"comments_url": "https://api.github.com/repos/zhenxun-org/zhenxun_bot_plugins_index/commits/2ed61284873c526802752b12a3fd3b5e1a59d948/comments",
"author": {
"login": "zhenxunflow[bot]",
"id": 179375394,
"node_id": "BOT_kgDOCrENIg",
"avatar_url": "https://avatars.githubusercontent.com/in/978723?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/zhenxunflow%5Bbot%5D",
"html_url": "https://github.com/apps/zhenxunflow",
"followers_url": "https://api.github.com/users/zhenxunflow%5Bbot%5D/followers",
"following_url": "https://api.github.com/users/zhenxunflow%5Bbot%5D/following{/other_user}",
"gists_url": "https://api.github.com/users/zhenxunflow%5Bbot%5D/gists{/gist_id}",
"starred_url": "https://api.github.com/users/zhenxunflow%5Bbot%5D/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/zhenxunflow%5Bbot%5D/subscriptions",
"organizations_url": "https://api.github.com/users/zhenxunflow%5Bbot%5D/orgs",
"repos_url": "https://api.github.com/users/zhenxunflow%5Bbot%5D/repos",
"events_url": "https://api.github.com/users/zhenxunflow%5Bbot%5D/events{/privacy}",
"received_events_url": "https://api.github.com/users/zhenxunflow%5Bbot%5D/received_events",
"type": "Bot",
"user_view_type": "public",
"site_admin": false
},
"committer": {
"login": "web-flow",
"id": 19864447,
"node_id": "MDQ6VXNlcjE5ODY0NDQ3",
"avatar_url": "https://avatars.githubusercontent.com/u/19864447?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/web-flow",
"html_url": "https://github.com/web-flow",
"followers_url": "https://api.github.com/users/web-flow/followers",
"following_url": "https://api.github.com/users/web-flow/following{/other_user}",
"gists_url": "https://api.github.com/users/web-flow/gists{/gist_id}",
"starred_url": "https://api.github.com/users/web-flow/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/web-flow/subscriptions",
"organizations_url": "https://api.github.com/users/web-flow/orgs",
"repos_url": "https://api.github.com/users/web-flow/repos",
"events_url": "https://api.github.com/users/web-flow/events{/privacy}",
"received_events_url": "https://api.github.com/users/web-flow/received_events",
"type": "User",
"user_view_type": "public",
"site_admin": false
},
"parents": [
{
"sha": "5df26081d40e3000a7beedb73954d4df397c93fa",
"url": "https://api.github.com/repos/zhenxun-org/zhenxun_bot_plugins_index/commits/5df26081d40e3000a7beedb73954d4df397c93fa",
"html_url": "https://github.com/zhenxun-org/zhenxun_bot_plugins_index/commit/5df26081d40e3000a7beedb73954d4df397c93fa"
}
],
"stats": {
"total": 11,
"additions": 11,
"deletions": 0
},
"files": [
{
"sha": "3d98392c25d38f5d375b830aed6e2298e47e5601",
"filename": "plugins.json",
"status": "modified",
"additions": 11,
"deletions": 0,
"changes": 11,
"blob_url": "https://github.com/zhenxun-org/zhenxun_bot_plugins_index/blob/2ed61284873c526802752b12a3fd3b5e1a59d948/plugins.json",
"raw_url": "https://github.com/zhenxun-org/zhenxun_bot_plugins_index/raw/2ed61284873c526802752b12a3fd3b5e1a59d948/plugins.json",
"contents_url": "https://api.github.com/repos/zhenxun-org/zhenxun_bot_plugins_index/contents/plugins.json?ref=2ed61284873c526802752b12a3fd3b5e1a59d948",
"patch": "@@ -53,5 +53,16 @@\n \"plugin_type\": \"NORMAL\",\n \"is_dir\": true,\n \"github_url\": \"https://github.com/PackageInstaller/zhenxun_plugin_draw_painting/tree/master\"\n+ },\n+ \"AI全家桶\": {\n+ \"module\": \"zhipu_toolkit\",\n+ \"module_path\": \"zhipu_toolkit\",\n+ \"description\": \"AI全家桶，一次安装，到处使用，省时省力省心\",\n+ \"usage\": \"AI全家桶，一次安装，到处使用，省时省力省心\\n usage:\\n 生成图片 <prompt>\\n 生成视频 <prompt>\\n 清理我的会话: 用于清理你与AI的聊天记录\\n 或者与机器人聊天，\\n 例如；\\n @Bot抱抱\\n 小真寻老婆\",\n+ \"author\": \"molanp\",\n+ \"version\": \"0.1\",\n+ \"plugin_type\": \"NORMAL\",\n+ \"is_dir\": true,\n+ \"github_url\": \"https://github.com/molanp/zhenxun_plugin_zhipu_toolkit\"\n }\n }"
}
]
}

83
tests/response/plugin_store/zhenxun_bot_plugins_metadata.json
View File

@ -1,83 +0,0 @@
{
"type": "gh",
"name": "zhenxun-org/zhenxun_bot_plugins",
"version": "main",
"default": null,
"files": [
{
"type": "directory",
"name": "plugins",
"files": [
{
"type": "directory",
"name": "search_image",
"files": [
{
"type": "file",
"name": "__init__.py",
"hash": "a4Yp9HPoBzMwvnQDT495u0yYqTQWofkOyHxEi1FdVb0=",
"size": 3010
}
]
},
{
"type": "directory",
"name": "alapi",
"files": [
{
"type": "file",
"name": "__init__.py",
"hash": "ndDxtO0pAq3ZTb4RdqW7FTDgOGC/RjS1dnwdaQfT0uQ=",
"size": 284
},
{
"type": "file",
"name": "_data_source.py",
"hash": "KOLqtj4TQWWQco5bA4tWFc7A0z1ruMyDk1RiKeqJHRA=",
"size": 919
},
{
"type": "file",
"name": "comments_163.py",
"hash": "Q5pZsj1Pj+EJMdKYcPtLqejcXAWUQIoXVQG49PZPaSI=",
"size": 1593
},
{
"type": "file",
"name": "cover.py",
"hash": "QSjtcy0oVrjaRiAWZKmUJlp0L4DQqEcdYNmExNo9mgc=",
"size": 1438
},
{
"type": "file",
"name": "jitang.py",
"hash": "xh43Osxt0xogTH448gUMC+/DaSGmCFme8DWUqC25IbU=",
"size": 1411
},
{
"type": "file",
"name": "poetry.py",
"hash": "Aj2unoNQboj3/0LhIrYU+dCa5jvMdpjMYXYUayhjuz4=",
"size": 1530
}
]
},
{
"type": "directory",
"name": "bilibili_sub",
"files": [
{
"type": "file",
"name": "__init__.py",
"hash": "407DCgNFcZnuEK+d716j8EWrFQc4Nlxa35V3yemy3WQ=",
"size": 14293
}
]
}
]
}
],
"links": {
"stats": "https://data.jsdelivr.com/v1/stats/packages/gh/zhenxun-org/zhenxun_bot_plugins@main"
}
}

1324
tests/response/plugin_store/zhenxun_bot_plugins_tree.json
View File

File diff suppressed because it is too large Load Diff

101
tests/response/plugin_store/zhenxun_github_sub_commit.json
View File

@ -1,101 +0,0 @@
{
"sha": "f524632f78d27f9893beebdf709e0e7885cd08f1",
"node_id": "C_kwDOJAjBPdoAKGY1MjQ2MzJmNzhkMjdmOTg5M2JlZWJkZjcwOWUwZTc4ODVjZDA4ZjE",
"commit": {
"author": {
"name": "xuaner",
"email": "xuaner_wa@qq.com",
"date": "2024-11-18T18:17:15Z"
},
"committer": {
"name": "xuaner",
"email": "xuaner_wa@qq.com",
"date": "2024-11-18T18:17:15Z"
},
"message": "fix bug",
"tree": {
"sha": "b6b1b4f06cc869b9f38d7b51bdca3a2c575255e4",
"url": "https://api.github.com/repos/xuanerwa/zhenxun_github_sub/git/trees/b6b1b4f06cc869b9f38d7b51bdca3a2c575255e4"
},
"url": "https://api.github.com/repos/xuanerwa/zhenxun_github_sub/git/commits/f524632f78d27f9893beebdf709e0e7885cd08f1",
"comment_count": 0,
"verification": {
"verified": false,
"reason": "unsigned",
"signature": null,
"payload": null,
"verified_at": null
}
},
"url": "https://api.github.com/repos/xuanerwa/zhenxun_github_sub/commits/f524632f78d27f9893beebdf709e0e7885cd08f1",
"html_url": "https://github.com/xuanerwa/zhenxun_github_sub/commit/f524632f78d27f9893beebdf709e0e7885cd08f1",
"comments_url": "https://api.github.com/repos/xuanerwa/zhenxun_github_sub/commits/f524632f78d27f9893beebdf709e0e7885cd08f1/comments",
"author": {
"login": "xuanerwa",
"id": 58063798,
"node_id": "MDQ6VXNlcjU4MDYzNzk4",
"avatar_url": "https://avatars.githubusercontent.com/u/58063798?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/xuanerwa",
"html_url": "https://github.com/xuanerwa",
"followers_url": "https://api.github.com/users/xuanerwa/followers",
"following_url": "https://api.github.com/users/xuanerwa/following{/other_user}",
"gists_url": "https://api.github.com/users/xuanerwa/gists{/gist_id}",
"starred_url": "https://api.github.com/users/xuanerwa/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/xuanerwa/subscriptions",
"organizations_url": "https://api.github.com/users/xuanerwa/orgs",
"repos_url": "https://api.github.com/users/xuanerwa/repos",
"events_url": "https://api.github.com/users/xuanerwa/events{/privacy}",
"received_events_url": "https://api.github.com/users/xuanerwa/received_events",
"type": "User",
"user_view_type": "public",
"site_admin": false
},
"committer": {
"login": "xuanerwa",
"id": 58063798,
"node_id": "MDQ6VXNlcjU4MDYzNzk4",
"avatar_url": "https://avatars.githubusercontent.com/u/58063798?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/xuanerwa",
"html_url": "https://github.com/xuanerwa",
"followers_url": "https://api.github.com/users/xuanerwa/followers",
"following_url": "https://api.github.com/users/xuanerwa/following{/other_user}",
"gists_url": "https://api.github.com/users/xuanerwa/gists{/gist_id}",
"starred_url": "https://api.github.com/users/xuanerwa/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/xuanerwa/subscriptions",
"organizations_url": "https://api.github.com/users/xuanerwa/orgs",
"repos_url": "https://api.github.com/users/xuanerwa/repos",
"events_url": "https://api.github.com/users/xuanerwa/events{/privacy}",
"received_events_url": "https://api.github.com/users/xuanerwa/received_events",
"type": "User",
"user_view_type": "public",
"site_admin": false
},
"parents": [
{
"sha": "91e5e2c792e79193830441d555769aa54acd2d15",
"url": "https://api.github.com/repos/xuanerwa/zhenxun_github_sub/commits/91e5e2c792e79193830441d555769aa54acd2d15",
"html_url": "https://github.com/xuanerwa/zhenxun_github_sub/commit/91e5e2c792e79193830441d555769aa54acd2d15"
}
],
"stats": {
"total": 2,
"additions": 1,
"deletions": 1
},
"files": [
{
"sha": "764a5f7b81554c4c10d29486ea5d9105e505cec3",
"filename": "github_sub/__init__.py",
"status": "modified",
"additions": 1,
"deletions": 1,
"changes": 2,
"blob_url": "https://github.com/xuanerwa/zhenxun_github_sub/blob/f524632f78d27f9893beebdf709e0e7885cd08f1/github_sub%2F__init__.py",
"raw_url": "https://github.com/xuanerwa/zhenxun_github_sub/raw/f524632f78d27f9893beebdf709e0e7885cd08f1/github_sub%2F__init__.py",
"contents_url": "https://api.github.com/repos/xuanerwa/zhenxun_github_sub/contents/github_sub%2F__init__.py?ref=f524632f78d27f9893beebdf709e0e7885cd08f1",
"patch": "@@ -168,7 +168,7 @@ async def _(session: EventSession):\n # 推送\n @scheduler.scheduled_job(\n \"interval\",\n- seconds=base_config.get(\"CHECK_API_TIME\") if base_config.get(\"CHECK_TIME\") else 30,\n+ seconds=base_config.get(\"CHECK_API_TIME\") if base_config.get(\"CHECK_API_TIME\") else 30,\n )\n async def _():\n bots = nonebot.get_bots()"
}
]
}

23
tests/response/plugin_store/zhenxun_github_sub_metadata.json
View File

@ -1,23 +0,0 @@
{
"type": "gh",
"name": "xuanerwa/zhenxun_github_sub",
"version": "main",
"default": null,
"files": [
{
"type": "directory",
"name": "github_sub",
"files": [
{
"type": "file",
"name": "__init__.py",
"hash": "z1C5BBK0+atbDghbyRlF2xIDwk0HQdHM1yXQZkF7/t8=",
"size": 7551
}
]
}
],
"links": {
"stats": "https://data.jsdelivr.com/v1/stats/packages/gh/xuanerwa/zhenxun_github_sub@main"
}
}

38
tests/response/plugin_store/zhenxun_github_sub_tree.json
View File

@ -1,38 +0,0 @@
{
"sha": "438298b9e88f9dafa7020e99d7c7b4c98f93aea6",
"url": "https://api.github.com/repos/xuanerwa/zhenxun_github_sub/git/trees/438298b9e88f9dafa7020e99d7c7b4c98f93aea6",
"tree": [
{
"path": "LICENSE",
"mode": "100644",
"type": "blob",
"sha": "f288702d2fa16d3cdf0035b15a9fcbc552cd88e7",
"size": 35149,
"url": "https://api.github.com/repos/xuanerwa/zhenxun_github_sub/git/blobs/f288702d2fa16d3cdf0035b15a9fcbc552cd88e7"
},
{
"path": "README.md",
"mode": "100644",
"type": "blob",
"sha": "e974cfc9b973d4a041f03e693ea20563a933b7ca",
"size": 955,
"url": "https://api.github.com/repos/xuanerwa/zhenxun_github_sub/git/blobs/e974cfc9b973d4a041f03e693ea20563a933b7ca"
},
{
"path": "github_sub",
"mode": "040000",
"type": "tree",
"sha": "0f7d76bcf472e2ab0610fa542b067633d6e3ae7e",
"url": "https://api.github.com/repos/xuanerwa/zhenxun_github_sub/git/trees/0f7d76bcf472e2ab0610fa542b067633d6e3ae7e"
},
{
"path": "github_sub/__init__.py",
"mode": "100644",
"type": "blob",
"sha": "7d17fd49fe82fa3897afcef61b2c694ed93a4ba3",
"size": 7551,
"url": "https://api.github.com/repos/xuanerwa/zhenxun_github_sub/git/blobs/7d17fd49fe82fa3897afcef61b2c694ed93a4ba3"
}
],
"truncated": false
}

5
zhenxun/builtin_plugins/__init__.py
View File

@ -17,7 +17,7 @@ from zhenxun.models.user_console import UserConsole
from zhenxun.services.log import logger
from zhenxun.utils.decorator.shop import shop_register
from zhenxun.utils.manager.priority_manager import PriorityLifecycle
from zhenxun.utils.manager.resource_manager import ResourceManager
from zhenxun.utils.manager.zhenxun_repo_manager import ZhenxunRepoManager
from zhenxun.utils.platform import PlatformUtils
driver: Driver = nonebot.get_driver()
@ -85,7 +85,8 @@ from bag_users t1
@PriorityLifecycle.on_startup(priority=5)
async def _():
await ResourceManager.init_resources()
if not ZhenxunRepoManager.check_resources_exists():
await ZhenxunRepoManager.resources_update()
"""签到与用户的数据迁移"""
if goods_list := await GoodsInfo.filter(uuid__isnull=True).all():
for goods in goods_list:

17
zhenxun/builtin_plugins/admin/group_member_update/_data_source.py
View File

@ -104,25 +104,16 @@ class MemberUpdateManage:
exist_member_list.append(member.id)
if data_list[0]:
try:
await GroupInfoUser.bulk_create(data_list[0], 30)
await GroupInfoUser.bulk_create(
data_list[0], 30, ignore_conflicts=True
)
logger.debug(
f"创建用户数据 {len(data_list[0])} 条",
"更新群组成员信息",
target=group_id,
)
except Exception as e:
logger.error(
f"批量创建用户数据失败: {e}，开始进行逐个存储",
"更新群组成员信息",
)
for u in data_list[0]:
try:
await u.save()
except Exception as e:
logger.error(
f"创建用户 {u.user_name}({u.user_id}) 数据失败: {e}",
"更新群组成员信息",
)
logger.error("批量创建用户数据失败", "更新群组成员信息", e=e)
if data_list[1]:
await GroupInfoUser.bulk_update(data_list[1], ["user_name"], 30)
logger.debug(

60
zhenxun/builtin_plugins/auto_update/__init__.py
View File

@ -16,10 +16,6 @@ from nonebot_plugin_uninfo import Uninfo
from zhenxun.configs.utils import PluginExtraData
from zhenxun.services.log import logger
from zhenxun.utils.enum import PluginType
from zhenxun.utils.manager.resource_manager import (
DownloadResourceException,
ResourceManager,
)
from zhenxun.utils.message import MessageUtils
from ._data_source import UpdateManager
@ -32,15 +28,23 @@ __plugin_meta__ = PluginMetadata(
检查更新真寻最新版本，包括了自动更新
资源文件大小一般在130mb左右，除非必须更新一般仅更新代码文件
指令：
检查更新 [main|release|resource|webui] ?[-r]
检查更新 [main|release|resource|webui] ?[-r] ?[-f] ?[-z] ?[-t]
main: main分支
release: 最新release
resource: 资源文件
webui: webui文件
-r: 下载资源文件，一般在更新main或release时使用
-f: 强制更新，一般用于更新main时使用（仅git更新时有效）
-s: 更新源，为 git 或 ali（默认使用ali）
-z: 下载zip文件进行更新（仅git有效）
-t: 更新方式，git或download（默认使用git）
git: 使用git pull（推荐）
download: 通过commit hash比较文件后下载更新（仅git有效）
示例:
检查更新 main
检查更新 main -r
检查更新 main -f
检查更新 release -r
检查更新 resource
检查更新 webui
@ -57,6 +61,9 @@ _matcher = on_alconna(
"检查更新",
Args["ver_type?", ["main", "release", "resource", "webui"]],
Option("-r|--resource", action=store_true, help_text="下载资源文件"),
Option("-f|--force", action=store_true, help_text="强制更新"),
Option("-s", Args["source?", ["git", "ali"]], help_text="更新源"),
Option("-z|--zip", action=store_true, help_text="下载zip文件"),
),
priority=1,
block=True,
@ -71,30 +78,55 @@ async def _(
session: Uninfo,
ver_type: Match[str],
resource: Query[bool] = Query("resource", False),
force: Query[bool] = Query("force", False),
source: Query[str] = Query("source", "ali"),
zip: Query[bool] = Query("zip", False),
):
result = ""
await MessageUtils.build_message("正在进行检查更新...").send(reply_to=True)
if ver_type.result in {"main", "release"}:
ver_type_str = ver_type.result
source_str = source.result
if ver_type_str in {"main", "release"}:
if not ver_type.available:
result = await UpdateManager.check_version()
result += await UpdateManager.check_version()
logger.info("查看当前版本...", "检查更新", session=session)
await MessageUtils.build_message(result).finish()
try:
result = await UpdateManager.update(bot, session.user.id, ver_type.result)
result += await UpdateManager.update_zhenxun(
bot,
session.user.id,
ver_type_str, # type: ignore
force.result,
source_str, # type: ignore
zip.result,
)
except Exception as e:
logger.error("版本更新失败...", "检查更新", session=session, e=e)
await MessageUtils.build_message(f"更新版本失败...e: {e}").finish()
elif ver_type.result == "webui":
result = await UpdateManager.update_webui()
if zip.result:
source_str = None
try:
result += await UpdateManager.update_webui(
source_str, # type: ignore
"test",
True,
)
except Exception as e:
logger.error("WebUI更新失败...", "检查更新", session=session, e=e)
result += "\nWebUI更新错误..."
if resource.result or ver_type.result == "resource":
try:
await ResourceManager.init_resources(True)
result += "\n资源文件更新成功！"
except DownloadResourceException:
result += "\n资源更新下载失败..."
if zip.result:
source_str = None
result += await UpdateManager.update_resources(
source_str, # type: ignore
"main",
force.result,
)
except Exception as e:
logger.error("资源更新下载失败...", "检查更新", session=session, e=e)
result += "\n资源更新未知错误..."
result += "\n资源更新错误..."
if result:
await MessageUtils.build_message(result.strip()).finish()
await MessageUtils.build_message("更新版本失败...").finish()

365
zhenxun/builtin_plugins/auto_update/_data_source.py
View File

@ -1,170 +1,19 @@
import os
import shutil
import subprocess
import tarfile
import zipfile
from typing import Literal
from nonebot.adapters import Bot
from nonebot.utils import run_sync
from zhenxun.configs.path_config import DATA_PATH
from zhenxun.services.log import logger
from zhenxun.utils.github_utils import GithubUtils
from zhenxun.utils.github_utils.models import RepoInfo
from zhenxun.utils.http_utils import AsyncHttpx
from zhenxun.utils.manager.virtual_env_package_manager import VirtualEnvPackageManager
from zhenxun.utils.manager.zhenxun_repo_manager import (
ZhenxunRepoConfig,
ZhenxunRepoManager,
)
from zhenxun.utils.platform import PlatformUtils
from .config import (
BACKUP_PATH,
BASE_PATH,
BASE_PATH_STRING,
COMMAND,
DEFAULT_GITHUB_URL,
DOWNLOAD_GZ_FILE,
DOWNLOAD_ZIP_FILE,
PYPROJECT_FILE,
PYPROJECT_FILE_STRING,
PYPROJECT_LOCK_FILE,
PYPROJECT_LOCK_FILE_STRING,
RELEASE_URL,
REPLACE_FOLDERS,
REQ_TXT_FILE,
REQ_TXT_FILE_STRING,
TMP_PATH,
VERSION_FILE,
)
def install_requirement():
requirement_path = (REQ_TXT_FILE).absolute()
if not requirement_path.exists():
logger.debug(
f"没有找到zhenxun的requirement.txt,目标路径为{requirement_path}", COMMAND
)
return
try:
result = subprocess.run(
["pip", "install", "-r", str(requirement_path)],
check=True,
capture_output=True,
text=True,
)
logger.debug(f"成功安装真寻依赖，日志:\n{result.stdout}", COMMAND)
except subprocess.CalledProcessError as e:
logger.error(f"安装真寻依赖失败，错误:\n{e.stderr}", COMMAND, e=e)
@run_sync
def _file_handle(latest_version: str | None):
"""文件移动操作
参数:
latest_version: 版本号
"""
BACKUP_PATH.mkdir(exist_ok=True, parents=True)
logger.debug("开始解压文件压缩包...", COMMAND)
download_file = DOWNLOAD_GZ_FILE
if DOWNLOAD_GZ_FILE.exists():
tf = tarfile.open(DOWNLOAD_GZ_FILE)
else:
download_file = DOWNLOAD_ZIP_FILE
tf = zipfile.ZipFile(DOWNLOAD_ZIP_FILE)
tf.extractall(TMP_PATH)
logger.debug("解压文件压缩包完成...", COMMAND)
download_file_path = TMP_PATH / next(
x for x in os.listdir(TMP_PATH) if (TMP_PATH / x).is_dir()
)
_pyproject = download_file_path / PYPROJECT_FILE_STRING
_lock_file = download_file_path / PYPROJECT_LOCK_FILE_STRING
_req_file = download_file_path / REQ_TXT_FILE_STRING
extract_path = download_file_path / BASE_PATH_STRING
target_path = BASE_PATH
if PYPROJECT_FILE.exists():
logger.debug(f"移除备份文件: {PYPROJECT_FILE}", COMMAND)
shutil.move(PYPROJECT_FILE, BACKUP_PATH / PYPROJECT_FILE_STRING)
if PYPROJECT_LOCK_FILE.exists():
logger.debug(f"移除备份文件: {PYPROJECT_LOCK_FILE}", COMMAND)
shutil.move(PYPROJECT_LOCK_FILE, BACKUP_PATH / PYPROJECT_LOCK_FILE_STRING)
if REQ_TXT_FILE.exists():
logger.debug(f"移除备份文件: {REQ_TXT_FILE}", COMMAND)
shutil.move(REQ_TXT_FILE, BACKUP_PATH / REQ_TXT_FILE_STRING)
if _pyproject.exists():
logger.debug("移动文件: pyproject.toml", COMMAND)
shutil.move(_pyproject, PYPROJECT_FILE)
if _lock_file.exists():
logger.debug("移动文件: poetry.lock", COMMAND)
shutil.move(_lock_file, PYPROJECT_LOCK_FILE)
if _req_file.exists():
logger.debug("移动文件: requirements.txt", COMMAND)
shutil.move(_req_file, REQ_TXT_FILE)
for folder in REPLACE_FOLDERS:
"""移动指定文件夹"""
_dir = BASE_PATH / folder
_backup_dir = BACKUP_PATH / folder
if _backup_dir.exists():
logger.debug(f"删除备份文件夹 {_backup_dir}", COMMAND)
shutil.rmtree(_backup_dir)
if _dir.exists():
logger.debug(f"移动旧文件夹 {_dir}", COMMAND)
shutil.move(_dir, _backup_dir)
else:
logger.warning(f"文件夹 {_dir} 不存在，跳过删除", COMMAND)
for folder in REPLACE_FOLDERS:
src_folder_path = extract_path / folder
dest_folder_path = target_path / folder
if src_folder_path.exists():
logger.debug(
f"移动文件夹: {src_folder_path} -> {dest_folder_path}", COMMAND
)
shutil.move(src_folder_path, dest_folder_path)
else:
logger.debug(f"源文件夹不存在: {src_folder_path}", COMMAND)
if tf:
tf.close()
if download_file.exists():
logger.debug(f"删除下载文件: {download_file}", COMMAND)
download_file.unlink()
if extract_path.exists():
logger.debug(f"删除解压文件夹: {extract_path}", COMMAND)
shutil.rmtree(extract_path)
if TMP_PATH.exists():
shutil.rmtree(TMP_PATH)
if latest_version:
with open(VERSION_FILE, "w", encoding="utf8") as f:
f.write(f"__version__: {latest_version}")
install_requirement()
LOG_COMMAND = "AutoUpdate"
class UpdateManager:
@classmethod
async def update_webui(cls) -> str:
from zhenxun.builtin_plugins.web_ui.public.data_source import (
update_webui_assets,
)
WEBUI_PATH = DATA_PATH / "web_ui" / "public"
BACKUP_PATH = DATA_PATH / "web_ui" / "backup_public"
if WEBUI_PATH.exists():
if BACKUP_PATH.exists():
logger.debug(f"删除旧的备份webui文件夹 {BACKUP_PATH}", COMMAND)
shutil.rmtree(BACKUP_PATH)
WEBUI_PATH.rename(BACKUP_PATH)
try:
await update_webui_assets()
logger.info("更新webui成功...", COMMAND)
if BACKUP_PATH.exists():
logger.debug(f"删除旧的webui文件夹 {BACKUP_PATH}", COMMAND)
shutil.rmtree(BACKUP_PATH)
return "Webui更新成功！"
except Exception as e:
logger.error("更新webui失败...", COMMAND, e=e)
if BACKUP_PATH.exists():
logger.debug(f"恢复旧的webui文件夹 {BACKUP_PATH}", COMMAND)
BACKUP_PATH.rename(WEBUI_PATH)
raise e
return ""
@classmethod
async def check_version(cls) -> str:
"""检查更新版本
@ -173,75 +22,146 @@ class UpdateManager:
str: 更新信息
"""
cur_version = cls.__get_version()
data = await cls.__get_latest_data()
if not data:
release_data = await ZhenxunRepoManager.zhenxun_get_latest_releases_data()
if not release_data:
return "检查更新获取版本失败..."
return (
"检测到当前版本更新\n"
f"当前版本：{cur_version}\n"
f"最新版本：{data.get('name')}\n"
f"创建日期：{data.get('created_at')}\n"
f"更新内容：\n{data.get('body')}"
f"最新版本：{release_data.get('name')}\n"
f"创建日期：{release_data.get('created_at')}\n"
f"更新内容：\n{release_data.get('body')}"
)
@classmethod
async def update(cls, bot: Bot, user_id: str, version_type: str) -> str:
async def update_webui(
cls,
source: Literal["git", "ali"] | None,
branch: str = "dist",
force: bool = False,
):
"""更新WebUI
参数:
source: 更新源
branch: 分支
force: 是否强制更新
返回:
str: 返回消息
"""
if not source:
await ZhenxunRepoManager.webui_zip_update()
return "WebUI更新完成!"
result = await ZhenxunRepoManager.webui_git_update(
source,
branch=branch,
force=force,
)
if not result.success:
logger.error(f"WebUI更新失败...错误: {result.error_message}", LOG_COMMAND)
return f"WebUI更新失败...错误: {result.error_message}"
return "WebUI更新完成!"
@classmethod
async def update_resources(
cls,
source: Literal["git", "ali"] | None,
branch: str = "main",
force: bool = False,
) -> str:
"""更新资源
参数:
source: 更新源
branch: 分支
force: 是否强制更新
返回:
str: 返回消息
"""
if not source:
await ZhenxunRepoManager.resources_zip_update()
return "真寻资源更新完成!"
result = await ZhenxunRepoManager.resources_git_update(
source,
branch=branch,
force=force,
)
if not result.success:
logger.error(
f"真寻资源更新失败...错误: {result.error_message}", LOG_COMMAND
)
return f"真寻资源更新失败...错误: {result.error_message}"
return "真寻资源更新完成!"
@classmethod
async def update_zhenxun(
cls,
bot: Bot,
user_id: str,
version_type: Literal["main", "release"],
force: bool,
source: Literal["git", "ali"],
zip: bool,
) -> str:
"""更新操作
参数:
bot: Bot
user_id: 用户id
version_type: 更新版本类型
force: 是否强制更新
source: 更新源
zip: 是否下载zip文件
update_type: 更新方式
返回:
str | None: 返回消息
"""
logger.info("开始下载真寻最新版文件....", COMMAND)
cur_version = cls.__get_version()
url = None
new_version = None
repo_info = GithubUtils.parse_github_url(DEFAULT_GITHUB_URL)
if version_type in {"main"}:
repo_info.branch = version_type
new_version = await cls.__get_version_from_repo(repo_info)
if new_version:
new_version = new_version.split(":")[-1].strip()
url = await repo_info.get_archive_download_urls()
elif version_type == "release":
data = await cls.__get_latest_data()
if not data:
return "获取更新版本失败..."
new_version = data.get("name", "")
url = await repo_info.get_release_source_download_urls_tgz(new_version)
if not url:
return "获取版本下载链接失败..."
if TMP_PATH.exists():
logger.debug(f"删除临时文件夹 {TMP_PATH}", COMMAND)
shutil.rmtree(TMP_PATH)
logger.debug(
f"开始更新版本：{cur_version} -> {new_version} | 下载链接：{url}",
COMMAND,
)
await PlatformUtils.send_superuser(
bot,
f"检测真寻已更新，版本更新：{cur_version} -> {new_version}\n开始更新...",
f"检测真寻已更新，当前版本：{cur_version}\n开始更新...",
user_id,
)
download_file = (
DOWNLOAD_GZ_FILE if version_type == "release" else DOWNLOAD_ZIP_FILE
)
if await AsyncHttpx.download_file(url, download_file, stream=True):
logger.debug("下载真寻最新版文件完成...", COMMAND)
await _file_handle(new_version)
result = "版本更新完成"
if zip:
new_version = await ZhenxunRepoManager.zhenxun_zip_update(version_type)
await PlatformUtils.send_superuser(
bot, "真寻更新完成，开始安装依赖...", user_id
)
await VirtualEnvPackageManager.install_requirement(
ZhenxunRepoConfig.REQUIREMENTS_FILE
)
return (
f"{result}\n"
f"版本: {cur_version} -> {new_version}\n"
f"版本更新完成！\n版本: {cur_version} -> {new_version}\n"
"请重新启动真寻以完成更新!"
)
else:
logger.debug("下载真寻最新版文件失败...", COMMAND)
return ""
result = await ZhenxunRepoManager.zhenxun_git_update(
source,
branch=version_type,
force=force,
)
if not result.success:
logger.error(
f"真寻版本更新失败...错误: {result.error_message}",
LOG_COMMAND,
)
return f"版本更新失败...错误: {result.error_message}"
await PlatformUtils.send_superuser(
bot, "真寻更新完成，开始安装依赖...", user_id
)
await VirtualEnvPackageManager.install_requirement(
ZhenxunRepoConfig.REQUIREMENTS_FILE
)
return (
f"版本更新完成！\n"
f"版本: {cur_version} -> {result.new_version}\n"
f"变更文件个数: {len(result.changed_files)}"
f"{'' if source == 'git' else '（阿里云更新不支持查看变更文件）'}\n"
"请重新启动真寻以完成更新!"
)
@classmethod
def __get_version(cls) -> str:
@ -251,44 +171,9 @@ class UpdateManager:
str: 当前版本号
"""
_version = "v0.0.0"
if VERSION_FILE.exists():
if text := VERSION_FILE.open(encoding="utf8").readline():
if ZhenxunRepoConfig.ZHENXUN_BOT_VERSION_FILE.exists():
if text := ZhenxunRepoConfig.ZHENXUN_BOT_VERSION_FILE.open(
encoding="utf8"
).readline():
_version = text.split(":")[-1].strip()
return _version
@classmethod
async def __get_latest_data(cls) -> dict:
"""获取最新版本信息
返回:
dict: 最新版本数据
"""
for _ in range(3):
try:
res = await AsyncHttpx.get(RELEASE_URL)
if res.status_code == 200:
return res.json()
except TimeoutError:
pass
except Exception as e:
logger.error("检查更新真寻获取版本失败", e=e)
return {}
@classmethod
async def __get_version_from_repo(cls, repo_info: RepoInfo) -> str:
"""从指定分支获取版本号
参数:
branch: 分支名称
返回:
str: 版本号
"""
version_url = await repo_info.get_raw_download_urls(path="__version__")
try:
res = await AsyncHttpx.get(version_url)
if res.status_code == 200:
return res.text.strip()
except Exception as e:
logger.error(f"获取 {repo_info.branch} 分支版本失败", e=e)
return "未知版本"

38
zhenxun/builtin_plugins/auto_update/config.py
View File

@ -1,38 +0,0 @@
from pathlib import Path
from zhenxun.configs.path_config import TEMP_PATH
DEFAULT_GITHUB_URL = "https://github.com/HibiKier/zhenxun_bot/tree/main"
RELEASE_URL = "https://api.github.com/repos/HibiKier/zhenxun_bot/releases/latest"
VERSION_FILE_STRING = "__version__"
VERSION_FILE = Path() / VERSION_FILE_STRING
PYPROJECT_FILE_STRING = "pyproject.toml"
PYPROJECT_FILE = Path() / PYPROJECT_FILE_STRING
PYPROJECT_LOCK_FILE_STRING = "poetry.lock"
PYPROJECT_LOCK_FILE = Path() / PYPROJECT_LOCK_FILE_STRING
REQ_TXT_FILE_STRING = "requirements.txt"
REQ_TXT_FILE = Path() / REQ_TXT_FILE_STRING
BASE_PATH_STRING = "zhenxun"
BASE_PATH = Path() / BASE_PATH_STRING
TMP_PATH = TEMP_PATH / "auto_update"
BACKUP_PATH = Path() / "backup"
DOWNLOAD_GZ_FILE_STRING = "download_latest_file.tar.gz"
DOWNLOAD_ZIP_FILE_STRING = "download_latest_file.zip"
DOWNLOAD_GZ_FILE = TMP_PATH / DOWNLOAD_GZ_FILE_STRING
DOWNLOAD_ZIP_FILE = TMP_PATH / DOWNLOAD_ZIP_FILE_STRING
REPLACE_FOLDERS = [
"builtin_plugins",
"services",
"utils",
"models",
"configs",
]
COMMAND = "检查更新"

5
zhenxun/builtin_plugins/hooks/auth_checker.py
View File

@ -148,6 +148,11 @@ async def get_plugin_and_user(
user = await with_timeout(
user_dao.safe_get_or_none(user_id=user_id), name="get_user"
)
except IntegrityError:
await asyncio.sleep(0.5)
plugin, user = await with_timeout(
asyncio.gather(plugin_task, user_task), name="get_plugin_and_user"
)
if not plugin:
raise PermissionExemption(f"插件:{module} 数据不存在，已跳过权限检查...")

2
zhenxun/builtin_plugins/plugin_store/__init__.py
View File

@ -84,7 +84,7 @@ async def _(session: EventSession):
try:
result = await StoreManager.get_plugins_info()
logger.info("查看插件列表", "插件商店", session=session)
await MessageUtils.build_message(result).send()
await MessageUtils.build_message([*result]).send()
except Exception as e:
logger.error(f"查看插件列表失败 e: {e}", "插件商店", session=session, e=e)
await MessageUtils.build_message("获取插件列表失败...").send()

385
zhenxun/builtin_plugins/plugin_store/data_source.py
View File

@ -1,19 +1,19 @@
from pathlib import Path
import random
import shutil
from aiocache import cached
import ujson as json
from zhenxun.builtin_plugins.auto_update.config import REQ_TXT_FILE_STRING
from zhenxun.builtin_plugins.plugin_store.models import StorePluginInfo
from zhenxun.configs.path_config import TEMP_PATH
from zhenxun.models.plugin_info import PluginInfo
from zhenxun.services.log import logger
from zhenxun.services.plugin_init import PluginInitManager
from zhenxun.utils.github_utils import GithubUtils
from zhenxun.utils.github_utils.models import RepoAPI
from zhenxun.utils.http_utils import AsyncHttpx
from zhenxun.utils.image_utils import BuildImage, ImageTemplate, RowStyle
from zhenxun.utils.manager.virtual_env_package_manager import VirtualEnvPackageManager
from zhenxun.utils.repo_utils import RepoFileManager
from zhenxun.utils.repo_utils.models import RepoFileInfo, RepoType
from zhenxun.utils.utils import is_number
from .config import (
@ -22,6 +22,7 @@ from .config import (
EXTRA_GITHUB_URL,
LOG_COMMAND,
)
from .exceptions import PluginStoreException
def row_style(column: str, text: str) -> RowStyle:
@ -40,73 +41,25 @@ def row_style(column: str, text: str) -> RowStyle:
return style
def install_requirement(plugin_path: Path):
requirement_files = ["requirement.txt", "requirements.txt"]
requirement_paths = [plugin_path / file for file in requirement_files]
if existing_requirements := next(
(path for path in requirement_paths if path.exists()), None
):
VirtualEnvPackageManager.install_requirement(existing_requirements)
class StoreManager:
@classmethod
async def get_github_plugins(cls) -> list[StorePluginInfo]:
"""获取github插件列表信息
返回:
list[StorePluginInfo]: 插件列表数据
"""
repo_info = GithubUtils.parse_github_url(DEFAULT_GITHUB_URL)
if await repo_info.update_repo_commit():
logger.info(f"获取最新提交: {repo_info.branch}", LOG_COMMAND)
else:
logger.warning(f"获取最新提交失败: {repo_info}", LOG_COMMAND)
default_github_url = await repo_info.get_raw_download_urls("plugins.json")
response = await AsyncHttpx.get(default_github_url, check_status_code=200)
if response.status_code == 200:
logger.info("获取github插件列表成功", LOG_COMMAND)
return [StorePluginInfo(**detail) for detail in json.loads(response.text)]
else:
logger.warning(
f"获取github插件列表失败: {response.status_code}", LOG_COMMAND
)
return []
@classmethod
async def get_extra_plugins(cls) -> list[StorePluginInfo]:
"""获取额外插件列表信息
返回:
list[StorePluginInfo]: 插件列表数据
"""
repo_info = GithubUtils.parse_github_url(EXTRA_GITHUB_URL)
if await repo_info.update_repo_commit():
logger.info(f"获取最新提交: {repo_info.branch}", LOG_COMMAND)
else:
logger.warning(f"获取最新提交失败: {repo_info}", LOG_COMMAND)
extra_github_url = await repo_info.get_raw_download_urls("plugins.json")
response = await AsyncHttpx.get(extra_github_url, check_status_code=200)
if response.status_code == 200:
return [StorePluginInfo(**detail) for detail in json.loads(response.text)]
else:
logger.warning(
f"获取github扩展插件列表失败: {response.status_code}", LOG_COMMAND
)
return []
@classmethod
@cached(60)
async def get_data(cls) -> list[StorePluginInfo]:
async def get_data(cls) -> tuple[list[StorePluginInfo], list[StorePluginInfo]]:
"""获取插件信息数据
返回:
list[StorePluginInfo]: 插件信息数据
tuple[list[StorePluginInfo], list[StorePluginInfo]]:
原生插件信息数据，第三方插件信息数据
"""
plugins = await cls.get_github_plugins()
extra_plugins = await cls.get_extra_plugins()
return [*plugins, *extra_plugins]
plugins = await RepoFileManager.get_file_content(
DEFAULT_GITHUB_URL, "plugins.json"
)
extra_plugins = await RepoFileManager.get_file_content(
EXTRA_GITHUB_URL, "plugins.json", "index"
)
return [StorePluginInfo(**plugin) for plugin in json.loads(plugins)], [
StorePluginInfo(**plugin) for plugin in json.loads(extra_plugins)
]
@classmethod
def version_check(cls, plugin_info: StorePluginInfo, suc_plugin: dict[str, str]):
@ -152,38 +105,105 @@ class StoreManager:
return await PluginInfo.filter(load_status=True).values_list(*args)
@classmethod
async def get_plugins_info(cls) -> BuildImage | str:
async def get_plugins_info(cls) -> list[BuildImage] | str:
"""插件列表
返回:
BuildImage | str: 返回消息
"""
plugin_list: list[StorePluginInfo] = await cls.get_data()
plugin_list, extra_plugin_list = await cls.get_data()
column_name = ["-", "ID", "名称", "简介", "作者", "版本", "类型"]
db_plugin_list = await cls.get_loaded_plugins("module", "version")
suc_plugin = {p[0]: (p[1] or "0.1") for p in db_plugin_list}
data_list = [
[
"已安装" if plugin_info.module in suc_plugin else "",
id,
plugin_info.name,
plugin_info.description,
plugin_info.author,
cls.version_check(plugin_info, suc_plugin),
plugin_info.plugin_type_name,
]
for id, plugin_info in enumerate(plugin_list)
index = 0
data_list = []
extra_data_list = []
for plugin_info in plugin_list:
data_list.append(
[
"已安装" if plugin_info.module in suc_plugin else "",
index,
plugin_info.name,
plugin_info.description,
plugin_info.author,
cls.version_check(plugin_info, suc_plugin),
plugin_info.plugin_type_name,
]
)
index += 1
for plugin_info in extra_plugin_list:
extra_data_list.append(
[
"已安装" if plugin_info.module in suc_plugin else "",
index,
plugin_info.name,
plugin_info.description,
plugin_info.author,
cls.version_check(plugin_info, suc_plugin),
plugin_info.plugin_type_name,
]
)
index += 1
return [
await ImageTemplate.table_page(
"原生插件列表",
"通过添加/移除插件 ID 来管理插件",
column_name,
data_list,
text_style=row_style,
),
await ImageTemplate.table_page(
"第三方插件列表",
"通过添加/移除插件 ID 来管理插件",
column_name,
extra_data_list,
text_style=row_style,
),
]
return await ImageTemplate.table_page(
"插件列表",
"通过添加/移除插件 ID 来管理插件",
column_name,
data_list,
text_style=row_style,
)
@classmethod
async def add_plugin(cls, plugin_id: str) -> str:
async def get_plugin_by_value(
cls, index_or_module: str, is_update: bool = False
) -> tuple[StorePluginInfo, bool]:
"""获取插件信息
参数:
index_or_module: 插件索引或模块名
is_update: 是否是更新插件
异常:
PluginStoreException: 插件不存在
PluginStoreException: 插件已安装
返回:
StorePluginInfo: 插件信息
bool: 是否是外部插件
"""
plugin_list, extra_plugin_list = await cls.get_data()
plugin_info = None
is_external = False
db_plugin_list = await cls.get_loaded_plugins("module")
plugin_key = await cls._resolve_plugin_key(index_or_module)
for p in plugin_list:
if p.module == plugin_key:
is_external = False
plugin_info = p
break
for p in extra_plugin_list:
if p.module == plugin_key:
is_external = True
plugin_info = p
break
if not plugin_info:
raise PluginStoreException(f"插件不存在: {plugin_key}")
if not is_update and plugin_info.module in [p[0] for p in db_plugin_list]:
raise PluginStoreException(f"插件 {plugin_info.name} 已安装，无需重复安装")
if plugin_info.module not in [p[0] for p in db_plugin_list] and is_update:
raise PluginStoreException(f"插件 {plugin_info.name} 未安装，无法更新")
return plugin_info, is_external
@classmethod
async def add_plugin(cls, index_or_module: str) -> str:
"""添加插件
参数:
@ -192,21 +212,9 @@ class StoreManager:
返回:
str: 返回消息
"""
plugin_list: list[StorePluginInfo] = await cls.get_data()
try:
plugin_key = await cls._resolve_plugin_key(plugin_id)
except ValueError as e:
return str(e)
db_plugin_list = await cls.get_loaded_plugins("module")
plugin_info = next((p for p in plugin_list if p.module == plugin_key), None)
if plugin_info is None:
return f"未找到插件 {plugin_key}"
if plugin_info.module in [p[0] for p in db_plugin_list]:
return f"插件 {plugin_info.name} 已安装，无需重复安装"
is_external = True
plugin_info, is_external = await cls.get_plugin_by_value(index_or_module)
if plugin_info.github_url is None:
plugin_info.github_url = DEFAULT_GITHUB_URL
is_external = False
version_split = plugin_info.version.split("-")
if len(version_split) > 1:
github_url_split = plugin_info.github_url.split("/tree/")
@ -228,90 +236,81 @@ class StoreManager:
is_dir: bool,
is_external: bool = False,
):
repo_api: RepoAPI
repo_info = GithubUtils.parse_github_url(github_url)
if await repo_info.update_repo_commit():
logger.info(f"获取最新提交: {repo_info.branch}", LOG_COMMAND)
else:
logger.warning(f"获取最新提交失败: {repo_info}", LOG_COMMAND)
logger.debug(f"成功获取仓库信息: {repo_info}", LOG_COMMAND)
for repo_api in GithubUtils.iter_api_strategies():
try:
await repo_api.parse_repo_info(repo_info)
break
except Exception as e:
logger.warning(
f"获取插件文件失败 | API类型: {repo_api.strategy}",
LOG_COMMAND,
e=e,
)
continue
else:
raise ValueError("所有API获取插件文件失败，请检查网络连接")
if module_path == ".":
module_path = ""
"""安装插件
参数:
github_url: 仓库地址
module_path: 模块路径
is_dir: 是否是文件夹
is_external: 是否是外部仓库
"""
repo_type = RepoType.GITHUB if is_external else None
replace_module_path = module_path.replace(".", "/")
files = repo_api.get_files(
module_path=replace_module_path + ("" if is_dir else ".py"),
is_dir=is_dir,
)
download_urls = [await repo_info.get_raw_download_urls(file) for file in files]
base_path = BASE_PATH / "plugins" if is_external else BASE_PATH
base_path = base_path if module_path else base_path / repo_info.repo
download_paths: list[Path | str] = [base_path / file for file in files]
logger.debug(f"插件下载路径: {download_paths}", LOG_COMMAND)
result = await AsyncHttpx.gather_download_file(download_urls, download_paths)
for _id, success in enumerate(result):
if not success:
break
if is_dir:
files = await RepoFileManager.list_directory_files(
github_url, replace_module_path, repo_type=repo_type
)
else:
# 安装依赖
plugin_path = base_path / "/".join(module_path.split("."))
try:
req_files = repo_api.get_files(
f"{replace_module_path}/{REQ_TXT_FILE_STRING}", False
files = [RepoFileInfo(path=f"{replace_module_path}.py", is_dir=False)]
local_path = BASE_PATH / "plugins" if is_external else BASE_PATH
files = [file for file in files if not file.is_dir]
download_files = [(file.path, local_path / file.path) for file in files]
await RepoFileManager.download_files(
github_url, download_files, repo_type=repo_type
)
requirement_paths = [
file
for file in files
if file.path.endswith("requirement.txt")
or file.path.endswith("requirements.txt")
]
is_install_req = False
for requirement_path in requirement_paths:
requirement_file = local_path / requirement_path.path
if requirement_file.exists():
is_install_req = True
await VirtualEnvPackageManager.install_requirement(requirement_file)
if not is_install_req:
# 从仓库根目录查找文件
rand = random.randint(1, 10000)
requirement_path = TEMP_PATH / f"plugin_store_{rand}_req.txt"
requirements_path = TEMP_PATH / f"plugin_store_{rand}_reqs.txt"
await RepoFileManager.download_files(
github_url,
[
("requirement.txt", requirement_path),
("requirements.txt", requirements_path),
],
repo_type=repo_type,
ignore_error=True,
)
if requirement_path.exists():
logger.info(
f"开始安装插件 {module_path} 依赖文件: {requirement_path}",
LOG_COMMAND,
)
req_files.extend(
repo_api.get_files(f"{replace_module_path}/requirement.txt", False)
await VirtualEnvPackageManager.install_requirement(requirement_path)
if requirements_path.exists():
logger.info(
f"开始安装插件 {module_path} 依赖文件: {requirements_path}",
LOG_COMMAND,
)
logger.debug(f"获取插件依赖文件列表: {req_files}", LOG_COMMAND)
req_download_urls = [
await repo_info.get_raw_download_urls(file) for file in req_files
]
req_paths: list[Path | str] = [plugin_path / file for file in req_files]
logger.debug(f"插件依赖文件下载路径: {req_paths}", LOG_COMMAND)
if req_files:
result = await AsyncHttpx.gather_download_file(
req_download_urls, req_paths
)
for success in result:
if not success:
raise Exception("插件依赖文件下载失败")
logger.debug(f"插件依赖文件列表: {req_paths}", LOG_COMMAND)
install_requirement(plugin_path)
except ValueError as e:
logger.warning("未获取到依赖文件路径...", e=e)
return True
raise Exception("插件下载失败...")
await VirtualEnvPackageManager.install_requirement(requirements_path)
@classmethod
async def remove_plugin(cls, plugin_id: str) -> str:
async def remove_plugin(cls, index_or_module: str) -> str:
"""移除插件
参数:
plugin_id: 插件id或模块名
index_or_module: 插件id或模块名
返回:
str: 返回消息
"""
plugin_list: list[StorePluginInfo] = await cls.get_data()
try:
plugin_key = await cls._resolve_plugin_key(plugin_id)
except ValueError as e:
return str(e)
plugin_info = next((p for p in plugin_list if p.module == plugin_key), None)
if plugin_info is None:
return f"未找到插件 {plugin_key}"
plugin_info, _ = await cls.get_plugin_by_value(index_or_module)
path = BASE_PATH
if plugin_info.github_url:
path = BASE_PATH / "plugins"
@ -339,12 +338,13 @@ class StoreManager:
返回:
BuildImage | str: 返回消息
"""
plugin_list: list[StorePluginInfo] = await cls.get_data()
plugin_list, extra_plugin_list = await cls.get_data()
all_plugin_list = plugin_list + extra_plugin_list
db_plugin_list = await cls.get_loaded_plugins("module", "version")
suc_plugin = {p[0]: (p[1] or "Unknown") for p in db_plugin_list}
filtered_data = [
(id, plugin_info)
for id, plugin_info in enumerate(plugin_list)
for id, plugin_info in enumerate(all_plugin_list)
if plugin_name_or_author.lower() in plugin_info.name.lower()
or plugin_name_or_author.lower() in plugin_info.author.lower()
]
@ -373,35 +373,24 @@ class StoreManager:
)
@classmethod
async def update_plugin(cls, plugin_id: str) -> str:
async def update_plugin(cls, index_or_module: str) -> str:
"""更新插件
参数:
plugin_id: 插件id
index_or_module: 插件id
返回:
str: 返回消息
"""
plugin_list: list[StorePluginInfo] = await cls.get_data()
try:
plugin_key = await cls._resolve_plugin_key(plugin_id)
except ValueError as e:
return str(e)
plugin_info = next((p for p in plugin_list if p.module == plugin_key), None)
if plugin_info is None:
return f"未找到插件 {plugin_key}"
plugin_info, is_external = await cls.get_plugin_by_value(index_or_module, True)
logger.info(f"尝试更新插件 {plugin_info.name}", LOG_COMMAND)
db_plugin_list = await cls.get_loaded_plugins("module", "version")
suc_plugin = {p[0]: (p[1] or "Unknown") for p in db_plugin_list}
if plugin_info.module not in [p[0] for p in db_plugin_list]:
return f"插件 {plugin_info.name} 未安装，无法更新"
logger.debug(f"当前插件列表: {suc_plugin}", LOG_COMMAND)
if cls.check_version_is_new(plugin_info, suc_plugin):
return f"插件 {plugin_info.name} 已是最新版本"
is_external = True
if plugin_info.github_url is None:
plugin_info.github_url = DEFAULT_GITHUB_URL
is_external = False
await cls.install_plugin_with_repo(
plugin_info.github_url,
plugin_info.module_path,
@ -420,8 +409,9 @@ class StoreManager:
返回:
str: 返回消息
"""
plugin_list: list[StorePluginInfo] = await cls.get_data()
plugin_name_list = [p.name for p in plugin_list]
plugin_list, extra_plugin_list = await cls.get_data()
all_plugin_list = plugin_list + extra_plugin_list
plugin_name_list = [p.name for p in all_plugin_list]
update_failed_list = []
update_success_list = []
result = "--已更新{}个插件 {}个失败 {}个成功--"
@ -492,22 +482,25 @@ class StoreManager:
plugin_id: module，id或插件名称
异常:
ValueError: 插件不存在
ValueError: 插件不存在
PluginStoreException: 插件不存在
PluginStoreException: 插件不存在
返回:
str: 插件模块名
"""
plugin_list: list[StorePluginInfo] = await cls.get_data()
plugin_list, extra_plugin_list = await cls.get_data()
all_plugin_list = plugin_list + extra_plugin_list
if is_number(plugin_id):
idx = int(plugin_id)
if idx < 0 or idx >= len(plugin_list):
raise ValueError("插件ID不存在...")
return plugin_list[idx].module
if idx < 0 or idx >= len(all_plugin_list):
raise PluginStoreException("插件ID不存在...")
return all_plugin_list[idx].module
elif isinstance(plugin_id, str):
result = (
None if plugin_id not in [v.module for v in plugin_list] else plugin_id
) or next(v for v in plugin_list if v.name == plugin_id).module
None
if plugin_id not in [v.module for v in all_plugin_list]
else plugin_id
) or next(v for v in all_plugin_list if v.name == plugin_id).module
if not result:
raise ValueError("插件 Module / 名称 不存在...")
raise PluginStoreException("插件 Module / 名称 不存在...")
return result

6
zhenxun/builtin_plugins/plugin_store/exceptions.py Normal file
View File

@ -0,0 +1,6 @@
class PluginStoreException(Exception):
def __init__(self, message: str):
self.message = message
def __str__(self):
return self.message

7
zhenxun/builtin_plugins/web_ui/api/configure/__init__.py
View File

@ -38,10 +38,11 @@ async def _(setting: Setting) -> Result:
password = Config.get_config("web-ui", "password")
if password or BotConfig.db_url:
return Result.fail("配置已存在，请先删除DB_URL内容和前端密码再进行设置。")
env_file = Path() / ".env.dev"
env_file = Path() / ".env.example"
if not env_file.exists():
return Result.fail("配置文件.env.dev不存在。")
return Result.fail("基础配置文件.env.example不存在。")
env_text = env_file.read_text(encoding="utf-8")
to_env_file = Path() / ".env.dev"
if setting.db_url:
if setting.db_url.startswith("sqlite"):
base_dir = Path().resolve()
@ -78,7 +79,7 @@ async def _(setting: Setting) -> Result:
if setting.username:
Config.set_config("web-ui", "username", setting.username)
Config.set_config("web-ui", "password", setting.password, True)
env_file.write_text(env_text, encoding="utf-8")
to_env_file.write_text(env_text, encoding="utf-8")
if BAT_FILE.exists():
for file in os.listdir(Path()):
if file.startswith(FILE_NAME):

4
zhenxun/builtin_plugins/web_ui/api/tabs/plugin_manage/__init__.py
View File

@ -229,9 +229,9 @@ async def _(payload: InstallDependenciesPayload) -> Result:
if not payload.dependencies:
return Result.fail("依赖列表不能为空")
if payload.handle_type == "install":
result = VirtualEnvPackageManager.install(payload.dependencies)
result = await VirtualEnvPackageManager.install(payload.dependencies)
else:
result = VirtualEnvPackageManager.uninstall(payload.dependencies)
result = await VirtualEnvPackageManager.uninstall(payload.dependencies)
return Result.ok(result)
except Exception as e:
logger.error(f"{router.prefix}/install_dependencies 调用错误", "WebUi", e=e)

4
zhenxun/builtin_plugins/web_ui/api/tabs/plugin_manage/store.py
View File

@ -25,10 +25,10 @@ async def _() -> Result[dict]:
require("plugin_store")
from zhenxun.builtin_plugins.plugin_store import StoreManager
data = await StoreManager.get_data()
plugin_list, extra_plugin_list = await StoreManager.get_data()
plugin_list = [
{**model_dump(plugin), "name": plugin.name, "id": idx}
for idx, plugin in enumerate(data)
for idx, plugin in enumerate(plugin_list + extra_plugin_list)
]
modules = await PluginInfo.filter(load_status=True).values_list(
"module", flat=True

10
zhenxun/builtin_plugins/web_ui/config.py
View File

@ -8,16 +8,6 @@ if sys.version_info >= (3, 11):
else:
from strenum import StrEnum
from zhenxun.configs.path_config import DATA_PATH, TEMP_PATH
WEBUI_STRING = "web_ui"
PUBLIC_STRING = "public"
WEBUI_DATA_PATH = DATA_PATH / WEBUI_STRING
PUBLIC_PATH = WEBUI_DATA_PATH / PUBLIC_STRING
TMP_PATH = TEMP_PATH / WEBUI_STRING
WEBUI_DIST_GITHUB_URL = "https://github.com/HibiKier/zhenxun_bot_webui/tree/dist"
app = nonebot.get_app()

29
zhenxun/builtin_plugins/web_ui/public/__init__.py
View File

@ -3,41 +3,38 @@ from fastapi.responses import FileResponse
from fastapi.staticfiles import StaticFiles
from zhenxun.services.log import logger
from ..config import PUBLIC_PATH
from .data_source import COMMAND_NAME, update_webui_assets
from zhenxun.utils.manager.zhenxun_repo_manager import ZhenxunRepoManager
router = APIRouter()
@router.get("/")
async def index():
return FileResponse(PUBLIC_PATH / "index.html")
return FileResponse(ZhenxunRepoManager.config.WEBUI_PATH / "index.html")
@router.get("/favicon.ico")
async def favicon():
return FileResponse(PUBLIC_PATH / "favicon.ico")
@router.get("/79edfa81f3308a9f.jfif")
async def _():
return FileResponse(PUBLIC_PATH / "79edfa81f3308a9f.jfif")
return FileResponse(ZhenxunRepoManager.config.WEBUI_PATH / "favicon.ico")
async def init_public(app: FastAPI):
try:
if not PUBLIC_PATH.exists():
folders = await update_webui_assets()
else:
folders = [x.name for x in PUBLIC_PATH.iterdir() if x.is_dir()]
if not ZhenxunRepoManager.check_webui_exists():
await ZhenxunRepoManager.webui_update(branch="test")
folders = [
x.name for x in ZhenxunRepoManager.config.WEBUI_PATH.iterdir() if x.is_dir()
]
app.include_router(router)
for pathname in folders:
logger.debug(f"挂载文件夹: {pathname}")
app.mount(
f"/{pathname}",
StaticFiles(directory=PUBLIC_PATH / pathname, check_dir=True),
StaticFiles(
directory=ZhenxunRepoManager.config.WEBUI_PATH / pathname,
check_dir=True,
),
name=f"public_{pathname}",
)
except Exception as e:
logger.error("初始化 WebUI资源 失败", COMMAND_NAME, e=e)
logger.error("初始化 WebUI资源 失败", "WebUI", e=e)

44
zhenxun/builtin_plugins/web_ui/public/data_source.py
View File

@ -1,44 +0,0 @@
from pathlib import Path
import shutil
import zipfile
from nonebot.utils import run_sync
from zhenxun.services.log import logger
from zhenxun.utils.github_utils import GithubUtils
from zhenxun.utils.http_utils import AsyncHttpx
from ..config import PUBLIC_PATH, TMP_PATH, WEBUI_DIST_GITHUB_URL
COMMAND_NAME = "WebUI资源管理"
async def update_webui_assets():
webui_assets_path = TMP_PATH / "webui_assets.zip"
download_url = await GithubUtils.parse_github_url(
WEBUI_DIST_GITHUB_URL
).get_archive_download_urls()
logger.info("开始下载 webui_assets 资源...", COMMAND_NAME)
if await AsyncHttpx.download_file(
download_url, webui_assets_path, follow_redirects=True
):
logger.info("下载 webui_assets 成功...", COMMAND_NAME)
return await _file_handle(webui_assets_path)
raise Exception("下载 webui_assets 失败", COMMAND_NAME)
@run_sync
def _file_handle(webui_assets_path: Path):
logger.debug("开始解压 webui_assets...", COMMAND_NAME)
if webui_assets_path.exists():
tf = zipfile.ZipFile(webui_assets_path)
tf.extractall(TMP_PATH)
logger.debug("解压 webui_assets 成功...", COMMAND_NAME)
else:
raise Exception("解压 webui_assets 失败，文件不存在...", COMMAND_NAME)
download_file_path = next(f for f in TMP_PATH.iterdir() if f.is_dir())
shutil.rmtree(PUBLIC_PATH, ignore_errors=True)
shutil.copytree(download_file_path / "dist", PUBLIC_PATH, dirs_exist_ok=True)
logger.debug("复制 webui_assets 成功...", COMMAND_NAME)
shutil.rmtree(TMP_PATH, ignore_errors=True)
return [x.name for x in PUBLIC_PATH.iterdir() if x.is_dir()]

12
zhenxun/services/db_context/__init__.py
View File

@ -1,6 +1,8 @@
import asyncio
from pathlib import Path
from urllib.parse import urlparse
import aiofiles
import nonebot
from nonebot.utils import is_coroutine_callable
from tortoise import Tortoise
@ -86,6 +88,7 @@ def get_config() -> dict:
**MYSQL_CONFIG,
}
elif parsed.scheme == "sqlite":
Path(parsed.path).parent.mkdir(parents=True, exist_ok=True)
config["connections"]["default"] = {
"engine": "tortoise.backends.sqlite",
"credentials": {
@ -100,6 +103,15 @@ def get_config() -> dict:
async def init():
global MODELS, SCRIPT_METHOD
env_example_file = Path() / ".env.example"
env_dev_file = Path() / ".env.dev"
if not env_dev_file.exists():
async with aiofiles.open(env_example_file, encoding="utf-8") as f:
env_text = await f.read()
async with aiofiles.open(env_dev_file, "w", encoding="utf-8") as f:
await f.write(env_text)
logger.info("已生成 .env.dev 文件，请根据 .env.example 文件配置进行配置")
MODELS = db_model.models
SCRIPT_METHOD = db_model.script_method
if not BotConfig.db_url:

1
zhenxun/utils/github_utils/func.py
View File

@ -57,6 +57,7 @@ async def get_fastest_release_formats() -> list[str]:
async def get_fastest_release_source_formats() -> list[str]:
"""获取最快的发行版源码下载地址格式"""
formats: dict[str, str] = {
"https://github.bibk.top": "https://github.bibk.top/{owner}/{repo}/releases/download/{version}/{filename}",
"https://codeload.github.com/": RELEASE_SOURCE_FORMAT,
"https://p.102333.xyz/": f"https://p.102333.xyz/{RELEASE_SOURCE_FORMAT}",
}

46
zhenxun/utils/github_utils/models.py
View File

@ -317,6 +317,20 @@ class AliyunFileInfo:
repository_id: str
"""仓库ID"""
@classmethod
async def get_client(cls) -> devops20210625Client:
"""获取阿里云客户端"""
config = open_api_models.Config(
access_key_id=Aliyun_AccessKey_ID,
access_key_secret=base64.b64decode(
Aliyun_Secret_AccessKey_encrypted.encode()
).decode(),
endpoint=ALIYUN_ENDPOINT,
region_id=ALIYUN_REGION,
)
return devops20210625Client(config)
@classmethod
async def get_file_content(
cls, file_path: str, repo: str, ref: str = "main"
@ -335,16 +349,8 @@ class AliyunFileInfo:
repository_id = ALIYUN_REPO_MAPPING.get(repo)
if not repository_id:
raise ValueError(f"未找到仓库 {repo} 对应的阿里云仓库ID")
config = open_api_models.Config(
access_key_id=Aliyun_AccessKey_ID,
access_key_secret=base64.b64decode(
Aliyun_Secret_AccessKey_encrypted.encode()
).decode(),
endpoint=ALIYUN_ENDPOINT,
region_id=ALIYUN_REGION,
)
client = devops20210625Client(config)
client = await cls.get_client()
request = devops_20210625_models.GetFileBlobsRequest(
organization_id=ALIYUN_ORG_ID,
@ -404,16 +410,7 @@ class AliyunFileInfo:
if not repository_id:
raise ValueError(f"未找到仓库 {repo} 对应的阿里云仓库ID")
config = open_api_models.Config(
access_key_id=Aliyun_AccessKey_ID,
access_key_secret=base64.b64decode(
Aliyun_Secret_AccessKey_encrypted.encode()
).decode(),
endpoint=ALIYUN_ENDPOINT,
region_id=ALIYUN_REGION,
)
client = devops20210625Client(config)
client = await cls.get_client()
request = devops_20210625_models.ListRepositoryTreeRequest(
organization_id=ALIYUN_ORG_ID,
@ -459,16 +456,7 @@ class AliyunFileInfo:
if not repository_id:
raise ValueError(f"未找到仓库 {repo} 对应的阿里云仓库ID")
config = open_api_models.Config(
access_key_id=Aliyun_AccessKey_ID,
access_key_secret=base64.b64decode(
Aliyun_Secret_AccessKey_encrypted.encode()
).decode(),
endpoint=ALIYUN_ENDPOINT,
region_id=ALIYUN_REGION,
)
client = devops20210625Client(config)
client = await cls.get_client()
request = devops_20210625_models.GetRepositoryCommitRequest(
organization_id=ALIYUN_ORG_ID,

87
zhenxun/utils/manager/resource_manager.py
View File

@ -1,87 +0,0 @@
import os
from pathlib import Path
import shutil
import zipfile
from zhenxun.configs.path_config import FONT_PATH
from zhenxun.services.log import logger
from zhenxun.utils.github_utils import GithubUtils
from zhenxun.utils.http_utils import AsyncHttpx
CMD_STRING = "ResourceManager"
class DownloadResourceException(Exception):
pass
class ResourceManager:
GITHUB_URL = "https://github.com/zhenxun-org/zhenxun-bot-resources/tree/main"
RESOURCE_PATH = Path() / "resources"
TMP_PATH = Path() / "_resource_tmp"
ZIP_FILE = TMP_PATH / "resources.zip"
UNZIP_PATH = None
@classmethod
async def init_resources(cls, force: bool = False):
if (FONT_PATH.exists() and os.listdir(FONT_PATH)) and not force:
return
if cls.TMP_PATH.exists():
logger.debug(
"resources临时文件夹已存在，移除resources临时文件夹", CMD_STRING
)
shutil.rmtree(cls.TMP_PATH)
cls.TMP_PATH.mkdir(parents=True, exist_ok=True)
try:
await cls.__download_resources()
cls.file_handle()
except Exception as e:
logger.error("获取resources资源包失败", CMD_STRING, e=e)
if cls.TMP_PATH.exists():
logger.debug("移除resources临时文件夹", CMD_STRING)
shutil.rmtree(cls.TMP_PATH)
@classmethod
def file_handle(cls):
if not cls.UNZIP_PATH:
return
cls.__recursive_folder(cls.UNZIP_PATH, "resources")
@classmethod
def __recursive_folder(cls, dir: Path, parent_path: str):
for file in dir.iterdir():
if file.is_dir():
cls.__recursive_folder(file, f"{parent_path}/{file.name}")
else:
res_file = Path(parent_path) / file.name
if res_file.exists():
res_file.unlink()
res_file.parent.mkdir(parents=True, exist_ok=True)
file.rename(res_file)
@classmethod
async def __download_resources(cls):
"""获取resources文件夹"""
repo_info = GithubUtils.parse_github_url(cls.GITHUB_URL)
url = await repo_info.get_archive_download_urls()
logger.debug("开始下载resources资源包...", CMD_STRING)
if not await AsyncHttpx.download_file(url, cls.ZIP_FILE, stream=True):
logger.error(
"下载resources资源包失败，请尝试重启重新下载或前往 "
"https://github.com/zhenxun-org/zhenxun-bot-resources 手动下载..."
)
raise DownloadResourceException("下载resources资源包失败...")
logger.debug("下载resources资源文件压缩包完成...", CMD_STRING)
tf = zipfile.ZipFile(cls.ZIP_FILE)
tf.extractall(cls.TMP_PATH)
logger.debug("解压文件压缩包完成...", CMD_STRING)
download_file_path = cls.TMP_PATH / next(
x for x in os.listdir(cls.TMP_PATH) if (cls.TMP_PATH / x).is_dir()
)
cls.UNZIP_PATH = download_file_path / "resources"
if tf:
tf.close()

26
zhenxun/utils/manager/virtual_env_package_manager.py
View File

@ -1,3 +1,4 @@
import asyncio
from pathlib import Path
import subprocess
from subprocess import CalledProcessError
@ -36,7 +37,7 @@ class VirtualEnvPackageManager:
)
@classmethod
def install(cls, package: list[str] | str):
async def install(cls, package: list[str] | str):
"""安装依赖包
参数:
@ -49,7 +50,8 @@ class VirtualEnvPackageManager:
command.append("install")
command.append(" ".join(package))
logger.info(f"执行虚拟环境安装包指令: {command}", LOG_COMMAND)
result = subprocess.run(
result = await asyncio.to_thread(
subprocess.run,
command,
check=True,
capture_output=True,
@ -65,7 +67,7 @@ class VirtualEnvPackageManager:
return e.stderr
@classmethod
def uninstall(cls, package: list[str] | str):
async def uninstall(cls, package: list[str] | str):
"""卸载依赖包
参数:
@ -79,7 +81,8 @@ class VirtualEnvPackageManager:
command.append("-y")
command.append(" ".join(package))
logger.info(f"执行虚拟环境卸载包指令: {command}", LOG_COMMAND)
result = subprocess.run(
result = await asyncio.to_thread(
subprocess.run,
command,
check=True,
capture_output=True,
@ -95,7 +98,7 @@ class VirtualEnvPackageManager:
return e.stderr
@classmethod
def update(cls, package: list[str] | str):
async def update(cls, package: list[str] | str):
"""更新依赖包
参数:
@ -109,7 +112,8 @@ class VirtualEnvPackageManager:
command.append("--upgrade")
command.append(" ".join(package))
logger.info(f"执行虚拟环境更新包指令: {command}", LOG_COMMAND)
result = subprocess.run(
result = await asyncio.to_thread(
subprocess.run,
command,
check=True,
capture_output=True,
@ -122,7 +126,7 @@ class VirtualEnvPackageManager:
return e.stderr
@classmethod
def install_requirement(cls, requirement_file: Path):
async def install_requirement(cls, requirement_file: Path):
"""安装依赖文件
参数:
@ -139,7 +143,8 @@ class VirtualEnvPackageManager:
command.append("-r")
command.append(str(requirement_file.absolute()))
logger.info(f"执行虚拟环境安装依赖文件指令: {command}", LOG_COMMAND)
result = subprocess.run(
result = await asyncio.to_thread(
subprocess.run,
command,
check=True,
capture_output=True,
@ -158,13 +163,14 @@ class VirtualEnvPackageManager:
return e.stderr
@classmethod
def list(cls) -> str:
async def list(cls) -> str:
"""列出已安装的依赖包"""
try:
command = cls.__get_command()
command.append("list")
logger.info(f"执行虚拟环境列出包指令: {command}", LOG_COMMAND)
result = subprocess.run(
result = await asyncio.to_thread(
subprocess.run,
command,
check=True,
capture_output=True,

556
zhenxun/utils/manager/zhenxun_repo_manager.py Normal file
View File

@ -0,0 +1,556 @@
"""
真寻仓库管理器
负责真寻主仓库的更新、版本检查、文件处理等功能
"""
import os
from pathlib import Path
import shutil
from typing import ClassVar, Literal
import zipfile
import aiofiles
from zhenxun.configs.path_config import DATA_PATH, TEMP_PATH
from zhenxun.services.log import logger
from zhenxun.utils.github_utils import GithubUtils
from zhenxun.utils.http_utils import AsyncHttpx
from zhenxun.utils.manager.virtual_env_package_manager import VirtualEnvPackageManager
from zhenxun.utils.repo_utils import AliyunRepoManager, GithubRepoManager
from zhenxun.utils.repo_utils.models import RepoUpdateResult
from zhenxun.utils.repo_utils.utils import check_git
LOG_COMMAND = "ZhenxunRepoManager"
class ZhenxunUpdateException(Exception):
"""资源下载异常"""
pass
class ZhenxunRepoConfig:
"""真寻仓库配置"""
# Zhenxun Bot 相关配置
ZHENXUN_BOT_GIT = "https://github.com/zhenxun-org/zhenxun_bot.git"
ZHENXUN_BOT_GITHUB_URL = "https://github.com/HibiKier/zhenxun_bot/tree/main"
ZHENXUN_BOT_DOWNLOAD_FILE_STRING = "zhenxun_bot.zip"
ZHENXUN_BOT_DOWNLOAD_FILE = TEMP_PATH / ZHENXUN_BOT_DOWNLOAD_FILE_STRING
ZHENXUN_BOT_UNZIP_PATH = TEMP_PATH / "zhenxun_bot"
ZHENXUN_BOT_CODE_PATH = Path() / "zhenxun"
ZHENXUN_BOT_RELEASES_API_URL = (
"https://api.github.com/repos/HibiKier/zhenxun_bot/releases/latest"
)
ZHENXUN_BOT_BACKUP_PATH = Path() / "backup"
# 需要替换的文件夹
ZHENXUN_BOT_UPDATE_FOLDERS: ClassVar[list[str]] = [
"zhenxun/builtin_plugins",
"zhenxun/services",
"zhenxun/utils",
"zhenxun/models",
"zhenxun/configs",
]
ZHENXUN_BOT_VERSION_FILE_STRING = "__version__"
ZHENXUN_BOT_VERSION_FILE = Path() / ZHENXUN_BOT_VERSION_FILE_STRING
# 备份杂项
BACKUP_FILES: ClassVar[list[str]] = [
"pyproject.toml",
"poetry.lock",
"requirements.txt",
".env.dev",
".env.example",
]
# WEB UI 相关配置
WEBUI_GIT = "https://github.com/HibiKier/zhenxun_bot_webui.git"
WEBUI_DIST_GITHUB_URL = "https://github.com/HibiKier/zhenxun_bot_webui/tree/dist"
WEBUI_DOWNLOAD_FILE_STRING = "webui_assets.zip"
WEBUI_DOWNLOAD_FILE = TEMP_PATH / WEBUI_DOWNLOAD_FILE_STRING
WEBUI_UNZIP_PATH = TEMP_PATH / "web_ui"
WEBUI_PATH = DATA_PATH / "web_ui" / "public"
WEBUI_BACKUP_PATH = DATA_PATH / "web_ui" / "backup_public"
# 资源管理相关配置
RESOURCE_GIT = "https://github.com/zhenxun-org/zhenxun-bot-resources.git"
RESOURCE_GITHUB_URL = (
"https://github.com/zhenxun-org/zhenxun-bot-resources/tree/main"
)
RESOURCE_ZIP_FILE_STRING = "resources.zip"
RESOURCE_ZIP_FILE = TEMP_PATH / RESOURCE_ZIP_FILE_STRING
RESOURCE_UNZIP_PATH = TEMP_PATH / "resources"
RESOURCE_PATH = Path() / "resources"
REQUIREMENTS_FILE_STRING = "requirements.txt"
REQUIREMENTS_FILE = Path() / REQUIREMENTS_FILE_STRING
PYPROJECT_FILE_STRING = "pyproject.toml"
PYPROJECT_FILE = Path() / PYPROJECT_FILE_STRING
PYPROJECT_LOCK_FILE_STRING = "poetry.lock"
PYPROJECT_LOCK_FILE = Path() / PYPROJECT_LOCK_FILE_STRING
class ZhenxunRepoManagerClass:
"""真寻仓库管理器"""
def __init__(self):
self.config = ZhenxunRepoConfig()
def __clear_folder(self, folder_path: Path):
"""
清空文件夹
参数:
folder_path: 文件夹路径
"""
if not folder_path.exists():
return
for filename in os.listdir(folder_path):
file_path = folder_path / filename
try:
if file_path.is_file():
os.unlink(file_path)
elif file_path.is_dir() and not filename.startswith("."):
shutil.rmtree(file_path)
except Exception as e:
logger.warning(f"无法删除 {file_path}", LOG_COMMAND, e=e)
def __copy_files(self, src_path: Path, dest_path: Path, incremental: bool = False):
"""
复制文件或文件夹
参数:
src_path: 源文件或文件夹路径
dest_path: 目标文件或文件夹路径
incremental: 是否增量复制
"""
if src_path.is_file():
shutil.copy(src_path, dest_path)
logger.debug(f"复制文件 {src_path} -> {dest_path}", LOG_COMMAND)
elif src_path.is_dir():
for filename in os.listdir(src_path):
file_path = src_path / filename
dest_file = dest_path / filename
dest_file.parent.mkdir(exist_ok=True, parents=True)
if file_path.is_file():
if dest_file.exists():
dest_file.unlink()
shutil.copy(file_path, dest_file)
logger.debug(f"复制文件 {file_path} -> {dest_file}", LOG_COMMAND)
elif file_path.is_dir():
if incremental:
self.__copy_files(file_path, dest_file, incremental=True)
else:
if dest_file.exists():
shutil.rmtree(dest_file, True)
shutil.copytree(file_path, dest_file)
logger.debug(
f"复制文件夹 {file_path} -> {dest_file}",
LOG_COMMAND,
)
# ==================== Zhenxun Bot 相关方法 ====================
async def zhenxun_get_version_from_repo(self) -> str:
"""从指定分支获取版本号
返回:
str: 版本号
"""
repo_info = GithubUtils.parse_github_url(self.config.ZHENXUN_BOT_GITHUB_URL)
version_url = await repo_info.get_raw_download_urls(
path=self.config.ZHENXUN_BOT_VERSION_FILE_STRING
)
try:
res = await AsyncHttpx.get(version_url)
if res.status_code == 200:
return res.text.strip()
except Exception as e:
logger.error(f"获取 {repo_info.branch} 分支版本失败", LOG_COMMAND, e=e)
return "未知版本"
async def zhenxun_write_version_file(self, version: str):
"""写入版本文件"""
async with aiofiles.open(
self.config.ZHENXUN_BOT_VERSION_FILE, "w", encoding="utf8"
) as f:
await f.write(f"__version__: {version}")
def __backup_zhenxun(self):
"""备份真寻文件"""
for filename in os.listdir(self.config.ZHENXUN_BOT_CODE_PATH):
file_path = self.config.ZHENXUN_BOT_CODE_PATH / filename
if file_path.exists():
self.__copy_files(
file_path,
self.config.ZHENXUN_BOT_BACKUP_PATH / filename,
True,
)
for filename in self.config.BACKUP_FILES:
file_path = Path() / filename
if file_path.exists():
self.__copy_files(
file_path,
self.config.ZHENXUN_BOT_BACKUP_PATH / filename,
)
async def zhenxun_get_latest_releases_data(self) -> dict:
"""获取真寻releases最新版本信息
返回:
dict: 最新版本数据
"""
try:
res = await AsyncHttpx.get(self.config.ZHENXUN_BOT_RELEASES_API_URL)
if res.status_code == 200:
return res.json()
except Exception as e:
logger.error("检查更新真寻获取版本失败", LOG_COMMAND, e=e)
return {}
async def zhenxun_download_zip(self, ver_type: Literal["main", "release"]) -> str:
"""下载真寻最新版文件
参数:
ver_type: 版本类型，main 为最新版，release 为最新release版
返回:
str: 版本号
"""
repo_info = GithubUtils.parse_github_url(self.config.ZHENXUN_BOT_GITHUB_URL)
if ver_type == "main":
download_url = await repo_info.get_archive_download_urls()
new_version = await self.zhenxun_get_version_from_repo()
else:
release_data = await self.zhenxun_get_latest_releases_data()
logger.debug(f"获取真寻RELEASES最新版本信息: {release_data}", LOG_COMMAND)
if not release_data:
raise ZhenxunUpdateException("获取真寻RELEASES最新版本失败...")
new_version = release_data.get("name", "")
download_url = await repo_info.get_release_source_download_urls_tgz(
new_version
)
if not download_url:
raise ZhenxunUpdateException("获取真寻最新版文件下载链接失败...")
if self.config.ZHENXUN_BOT_DOWNLOAD_FILE.exists():
self.config.ZHENXUN_BOT_DOWNLOAD_FILE.unlink()
if await AsyncHttpx.download_file(
download_url, self.config.ZHENXUN_BOT_DOWNLOAD_FILE, stream=True
):
logger.debug("下载真寻最新版文件完成...", LOG_COMMAND)
else:
raise ZhenxunUpdateException("下载真寻最新版文件失败...")
return new_version
async def zhenxun_unzip(self):
"""解压真寻最新版文件"""
if not self.config.ZHENXUN_BOT_DOWNLOAD_FILE.exists():
raise FileNotFoundError("真寻最新版文件不存在")
if self.config.ZHENXUN_BOT_UNZIP_PATH.exists():
shutil.rmtree(self.config.ZHENXUN_BOT_UNZIP_PATH)
tf = None
try:
tf = zipfile.ZipFile(self.config.ZHENXUN_BOT_DOWNLOAD_FILE)
tf.extractall(self.config.ZHENXUN_BOT_UNZIP_PATH)
logger.debug("解压Zhenxun Bot文件压缩包完成!", LOG_COMMAND)
self.__backup_zhenxun()
for filename in self.config.BACKUP_FILES:
self.__copy_files(
self.config.ZHENXUN_BOT_UNZIP_PATH / filename,
Path() / filename,
)
logger.debug("备份真寻更新文件完成!", LOG_COMMAND)
unzip_dir = next(self.config.ZHENXUN_BOT_UNZIP_PATH.iterdir())
for folder in self.config.ZHENXUN_BOT_UPDATE_FOLDERS:
self.__copy_files(unzip_dir / folder, Path() / folder)
logger.debug("移动真寻更新文件完成!", LOG_COMMAND)
if self.config.ZHENXUN_BOT_UNZIP_PATH.exists():
shutil.rmtree(self.config.ZHENXUN_BOT_UNZIP_PATH)
except Exception as e:
logger.error("解压真寻最新版文件失败...", LOG_COMMAND, e=e)
raise
finally:
if tf:
tf.close()
async def zhenxun_zip_update(self, ver_type: Literal["main", "release"]) -> str:
"""使用zip更新真寻
参数:
ver_type: 版本类型，main 为最新版，release 为最新release版
返回:
str: 版本号
"""
new_version = await self.zhenxun_download_zip(ver_type)
await self.zhenxun_unzip()
await self.zhenxun_write_version_file(new_version)
return new_version
async def zhenxun_git_update(
self, source: Literal["git", "ali"], branch: str = "main", force: bool = False
) -> RepoUpdateResult:
"""使用git或阿里云更新真寻
参数:
source: 更新源，git 为 git 更新，ali 为阿里云更新
branch: 分支名称
force: 是否强制更新
"""
if source == "git":
return await GithubRepoManager.update_via_git(
self.config.ZHENXUN_BOT_GIT,
Path(),
branch=branch,
force=force,
)
else:
return await AliyunRepoManager.update_via_git(
self.config.ZHENXUN_BOT_GIT,
Path(),
branch=branch,
force=force,
)
async def zhenxun_update(
self,
source: Literal["git", "ali"] = "ali",
branch: str = "main",
force: bool = False,
ver_type: Literal["main", "release"] = "main",
):
"""更新真寻
参数:
source: 更新源，git 为 git 更新，ali 为阿里云更新
branch: 分支名称
force: 是否强制更新
ver_type: 版本类型，main 为最新版，release 为最新release版
"""
if await check_git():
await self.zhenxun_git_update(source, branch, force)
logger.debug("使用git更新真寻!", LOG_COMMAND)
else:
await self.zhenxun_zip_update(ver_type)
logger.debug("使用zip更新真寻!", LOG_COMMAND)
async def install_requirements(self):
"""安装真寻依赖"""
await VirtualEnvPackageManager.install_requirement(
self.config.REQUIREMENTS_FILE
)
# ==================== 资源管理相关方法 ====================
def check_resources_exists(self) -> bool:
"""检查资源文件是否存在
返回:
bool: 是否存在
"""
if self.config.RESOURCE_PATH.exists():
font_path = self.config.RESOURCE_PATH / "font"
if font_path.exists() and os.listdir(font_path):
return True
return False
async def resources_download_zip(self):
"""下载资源文件"""
download_url = await GithubUtils.parse_github_url(
self.config.RESOURCE_GITHUB_URL
).get_archive_download_urls()
logger.debug("开始下载resources资源包...", LOG_COMMAND)
if await AsyncHttpx.download_file(
download_url, self.config.RESOURCE_ZIP_FILE, stream=True
):
logger.debug("下载resources资源文件压缩包成功!", LOG_COMMAND)
else:
raise ZhenxunUpdateException("下载resources资源包失败...")
async def resources_unzip(self):
"""解压资源文件"""
if not self.config.RESOURCE_ZIP_FILE.exists():
raise FileNotFoundError("资源文件压缩包不存在")
if self.config.RESOURCE_UNZIP_PATH.exists():
shutil.rmtree(self.config.RESOURCE_UNZIP_PATH)
tf = None
try:
tf = zipfile.ZipFile(self.config.RESOURCE_ZIP_FILE)
tf.extractall(self.config.RESOURCE_UNZIP_PATH)
logger.debug("解压文件压缩包完成...", LOG_COMMAND)
unzip_dir = next(self.config.RESOURCE_UNZIP_PATH.iterdir())
self.__copy_files(unzip_dir, self.config.RESOURCE_PATH, True)
logger.debug("复制资源文件完成!", LOG_COMMAND)
shutil.rmtree(self.config.RESOURCE_UNZIP_PATH, ignore_errors=True)
except Exception as e:
logger.error("解压资源文件失败...", LOG_COMMAND, e=e)
raise
finally:
if tf:
tf.close()
async def resources_zip_update(self):
"""使用zip更新资源文件"""
await self.resources_download_zip()
await self.resources_unzip()
async def resources_git_update(
self, source: Literal["git", "ali"], branch: str = "main", force: bool = False
) -> RepoUpdateResult:
"""使用git或阿里云更新资源文件
参数:
source: 更新源，git 为 git 更新，ali 为阿里云更新
branch: 分支名称
force: 是否强制更新
"""
if source == "git":
return await GithubRepoManager.update_via_git(
self.config.RESOURCE_GIT,
self.config.RESOURCE_PATH,
branch=branch,
force=force,
)
else:
return await AliyunRepoManager.update_via_git(
self.config.RESOURCE_GIT,
self.config.RESOURCE_PATH,
branch=branch,
force=force,
)
async def resources_update(
self,
source: Literal["git", "ali"] = "ali",
branch: str = "main",
force: bool = False,
):
"""更新资源文件
参数:
source: 更新源，git 为 git 更新，ali 为阿里云更新
branch: 分支名称
force: 是否强制更新
"""
if await check_git():
await self.resources_git_update(source, branch, force)
logger.debug("使用git更新资源文件!", LOG_COMMAND)
else:
await self.resources_zip_update()
logger.debug("使用zip更新资源文件!", LOG_COMMAND)
# ==================== Web UI 管理相关方法 ====================
def check_webui_exists(self) -> bool:
"""检查 Web UI 资源是否存在"""
return bool(
self.config.WEBUI_PATH.exists() and os.listdir(self.config.WEBUI_PATH)
)
async def webui_download_zip(self):
"""下载 WEBUI_ASSETS 资源"""
download_url = await GithubUtils.parse_github_url(
self.config.WEBUI_DIST_GITHUB_URL
).get_archive_download_urls()
logger.info("开始下载 WEBUI_ASSETS 资源...", LOG_COMMAND)
if await AsyncHttpx.download_file(
download_url, self.config.WEBUI_DOWNLOAD_FILE, follow_redirects=True
):
logger.info("下载 WEBUI_ASSETS 成功!", LOG_COMMAND)
else:
raise ZhenxunUpdateException("下载 WEBUI_ASSETS 失败", LOG_COMMAND)
def __backup_webui(self):
"""备份 WEBUI_ASSERT 资源"""
if self.config.WEBUI_PATH.exists():
if self.config.WEBUI_BACKUP_PATH.exists():
logger.debug(
f"删除旧的备份webui文件夹 {self.config.WEBUI_BACKUP_PATH}",
LOG_COMMAND,
)
shutil.rmtree(self.config.WEBUI_BACKUP_PATH)
shutil.copytree(self.config.WEBUI_PATH, self.config.WEBUI_BACKUP_PATH)
async def webui_unzip(self):
"""解压 WEBUI_ASSETS 资源
返回:
str: 更新结果
"""
if not self.config.WEBUI_DOWNLOAD_FILE.exists():
raise FileNotFoundError("webui文件压缩包不存在")
tf = None
try:
self.__backup_webui()
self.__clear_folder(self.config.WEBUI_PATH)
tf = zipfile.ZipFile(self.config.WEBUI_DOWNLOAD_FILE)
tf.extractall(self.config.WEBUI_UNZIP_PATH)
logger.debug("Web UI 解压文件压缩包完成...", LOG_COMMAND)
unzip_dir = next(self.config.WEBUI_UNZIP_PATH.iterdir())
self.__copy_files(unzip_dir, self.config.WEBUI_PATH)
logger.debug("Web UI 复制 WEBUI_ASSETS 成功!", LOG_COMMAND)
shutil.rmtree(self.config.WEBUI_UNZIP_PATH, ignore_errors=True)
except Exception as e:
if self.config.WEBUI_BACKUP_PATH.exists():
self.__copy_files(self.config.WEBUI_BACKUP_PATH, self.config.WEBUI_PATH)
logger.debug("恢复备份 WEBUI_ASSETS 成功!", LOG_COMMAND)
shutil.rmtree(self.config.WEBUI_BACKUP_PATH, ignore_errors=True)
logger.error("Web UI 更新失败", LOG_COMMAND, e=e)
raise
finally:
if tf:
tf.close()
async def webui_zip_update(self):
"""使用zip更新 Web UI"""
await self.webui_download_zip()
await self.webui_unzip()
async def webui_git_update(
self, source: Literal["git", "ali"], branch: str = "dist", force: bool = False
) -> RepoUpdateResult:
"""使用git或阿里云更新 Web UI
参数:
source: 更新源，git 为 git 更新，ali 为阿里云更新
branch: 分支名称
force: 是否强制更新
"""
if source == "git":
return await GithubRepoManager.update_via_git(
self.config.WEBUI_GIT,
self.config.WEBUI_PATH,
branch=branch,
force=force,
)
else:
return await AliyunRepoManager.update_via_git(
self.config.WEBUI_GIT,
self.config.WEBUI_PATH,
branch=branch,
force=force,
)
async def webui_update(
self,
source: Literal["git", "ali"] = "ali",
branch: str = "dist",
force: bool = False,
):
"""更新 Web UI
参数:
source: 更新源，git 为 git 更新，ali 为阿里云更新
"""
if await check_git():
await self.webui_git_update(source, branch, force)
logger.debug("使用git更新Web UI!", LOG_COMMAND)
else:
await self.webui_zip_update()
logger.debug("使用zip更新Web UI!", LOG_COMMAND)
ZhenxunRepoManager = ZhenxunRepoManagerClass()

60
zhenxun/utils/repo_utils/__init__.py Normal file
View File

@ -0,0 +1,60 @@
"""
仓库管理工具，用于操作GitHub和阿里云CodeUp项目的更新和文件下载
"""
from .aliyun_manager import AliyunCodeupManager
from .base_manager import BaseRepoManager
from .config import AliyunCodeupConfig, GithubConfig, RepoConfig
from .exceptions import (
ApiRateLimitError,
AuthenticationError,
ConfigError,
FileNotFoundError,
NetworkError,
RepoDownloadError,
RepoManagerError,
RepoNotFoundError,
RepoUpdateError,
)
from .file_manager import RepoFileManager as RepoFileManagerClass
from .github_manager import GithubManager
from .models import (
FileDownloadResult,
RepoCommitInfo,
RepoFileInfo,
RepoType,
RepoUpdateResult,
)
from .utils import check_git, filter_files, glob_to_regex, run_git_command
GithubRepoManager = GithubManager()
AliyunRepoManager = AliyunCodeupManager()
RepoFileManager = RepoFileManagerClass()
__all__ = [
"AliyunCodeupConfig",
"AliyunRepoManager",
"ApiRateLimitError",
"AuthenticationError",
"BaseRepoManager",
"ConfigError",
"FileDownloadResult",
"FileNotFoundError",
"GithubConfig",
"GithubRepoManager",
"NetworkError",
"RepoCommitInfo",
"RepoConfig",
"RepoDownloadError",
"RepoFileInfo",
"RepoFileManager",
"RepoManagerError",
"RepoNotFoundError",
"RepoType",
"RepoUpdateError",
"RepoUpdateResult",
"check_git",
"filter_files",
"glob_to_regex",
"run_git_command",
]

557
zhenxun/utils/repo_utils/aliyun_manager.py Normal file
View File

@ -0,0 +1,557 @@
"""
阿里云CodeUp仓库管理工具
"""
import asyncio
from collections.abc import Callable
from datetime import datetime
from pathlib import Path
from aiocache import cached
from zhenxun.services.log import logger
from zhenxun.utils.github_utils.models import AliyunFileInfo
from .base_manager import BaseRepoManager
from .config import LOG_COMMAND, RepoConfig
from .exceptions import (
AuthenticationError,
FileNotFoundError,
RepoDownloadError,
RepoNotFoundError,
RepoUpdateError,
)
from .models import (
FileDownloadResult,
RepoCommitInfo,
RepoFileInfo,
RepoType,
RepoUpdateResult,
)
class AliyunCodeupManager(BaseRepoManager):
"""阿里云CodeUp仓库管理工具"""
def __init__(self, config: RepoConfig | None = None):
"""
初始化阿里云CodeUp仓库管理工具
Args:
config: 配置，如果为None则使用默认配置
"""
super().__init__(config)
self._client = None
async def update_repo(
self,
repo_url: str,
local_path: Path,
branch: str = "main",
include_patterns: list[str] | None = None,
exclude_patterns: list[str] | None = None,
) -> RepoUpdateResult:
"""
更新阿里云CodeUp仓库
Args:
repo_url: 仓库URL或名称
local_path: 本地保存路径
branch: 分支名称
include_patterns: 包含的文件模式列表，如 ["*.py", "docs/*.md"]
exclude_patterns: 排除的文件模式列表，如 ["__pycache__/*", "*.pyc"]
Returns:
RepoUpdateResult: 更新结果
"""
try:
# 检查配置
self._check_config()
# 获取仓库名称（从URL中提取）
repo_url = repo_url.split("/tree/")[0].split("/")[-1].replace(".git", "")
# 获取仓库最新提交ID
newest_commit = await self._get_newest_commit(repo_url, branch)
# 创建结果对象
result = RepoUpdateResult(
repo_type=RepoType.ALIYUN,
repo_name=repo_url.split("/tree/")[0]
.split("/")[-1]
.replace(".git", ""),
owner=self.config.aliyun_codeup.organization_id,
old_version="", # 将在后面更新
new_version=newest_commit,
)
old_version = await self.read_version_file(local_path)
result.old_version = old_version
# 如果版本相同，则无需更新
if old_version == newest_commit:
result.success = True
logger.debug(
f"仓库 {repo_url.split('/')[-1].replace('.git', '')}"
f" 已是最新版本: {newest_commit[:8]}",
LOG_COMMAND,
)
return result
# 确保本地目录存在
local_path.mkdir(parents=True, exist_ok=True)
# 获取仓库名称（从URL中提取）
repo_name = repo_url.split("/tree/")[0].split("/")[-1].replace(".git", "")
# 获取变更的文件列表
changed_files = await self._get_changed_files(
repo_name, old_version or None, newest_commit
)
# 过滤文件
if include_patterns or exclude_patterns:
from .utils import filter_files
changed_files = filter_files(
changed_files, include_patterns, exclude_patterns
)
result.changed_files = changed_files
# 下载变更的文件
for file_path in changed_files:
try:
local_file_path = local_path / file_path
await self._download_file(
repo_name, file_path, local_file_path, newest_commit
)
except Exception as e:
logger.error(f"下载文件 {file_path} 失败", LOG_COMMAND, e=e)
# 更新版本文件
await self.write_version_file(local_path, newest_commit)
result.success = True
return result
except RepoUpdateError as e:
logger.error(f"更新仓库失败: {e}")
# 从URL中提取仓库名称
repo_name = repo_url.split("/tree/")[0].split("/")[-1].replace(".git", "")
return RepoUpdateResult(
repo_type=RepoType.ALIYUN,
repo_name=repo_name,
owner=self.config.aliyun_codeup.organization_id,
old_version="",
new_version="",
error_message=str(e),
)
except Exception as e:
logger.error(f"更新仓库失败: {e}")
# 从URL中提取仓库名称
repo_name = repo_url.split("/tree/")[0].split("/")[-1].replace(".git", "")
return RepoUpdateResult(
repo_type=RepoType.ALIYUN,
repo_name=repo_name,
owner=self.config.aliyun_codeup.organization_id,
old_version="",
new_version="",
error_message=str(e),
)
async def download_file(
self,
repo_url: str,
file_path: str,
local_path: Path,
branch: str = "main",
) -> FileDownloadResult:
"""
从阿里云CodeUp下载单个文件
Args:
repo_url: 仓库URL或名称
file_path: 文件在仓库中的路径
local_path: 本地保存路径
branch: 分支名称
Returns:
FileDownloadResult: 下载结果
"""
try:
# 检查配置
self._check_config()
# 获取仓库名称（从URL中提取）
repo_identifier = (
repo_url.split("/tree/")[0].split("/")[-1].replace(".git", "")
)
# 创建结果对象
result = FileDownloadResult(
repo_type=RepoType.ALIYUN,
repo_name=repo_url.split("/tree/")[0]
.split("/")[-1]
.replace(".git", ""),
file_path=file_path,
version=branch,
)
# 确保本地目录存在
Path(local_path).parent.mkdir(parents=True, exist_ok=True)
# 下载文件
file_size = await self._download_file(
repo_identifier, file_path, local_path, branch
)
result.success = True
result.file_size = file_size
return result
except RepoDownloadError as e:
logger.error(f"下载文件失败: {e}")
# 从URL中提取仓库名称
repo_name = repo_url.split("/tree/")[0].split("/")[-1].replace(".git", "")
return FileDownloadResult(
repo_type=RepoType.ALIYUN,
repo_name=repo_name,
file_path=file_path,
version=branch,
error_message=str(e),
)
except Exception as e:
logger.error(f"下载文件失败: {e}")
# 从URL中提取仓库名称
repo_name = repo_url.split("/tree/")[0].split("/")[-1].replace(".git", "")
return FileDownloadResult(
repo_type=RepoType.ALIYUN,
repo_name=repo_name,
file_path=file_path,
version=branch,
error_message=str(e),
)
async def get_file_list(
self,
repo_url: str,
dir_path: str = "",
branch: str = "main",
recursive: bool = False,
) -> list[RepoFileInfo]:
"""
获取仓库文件列表
Args:
repo_url: 仓库URL或名称
dir_path: 目录路径，空字符串表示仓库根目录
branch: 分支名称
recursive: 是否递归获取子目录
Returns:
list[RepoFileInfo]: 文件信息列表
"""
try:
# 检查配置
self._check_config()
# 获取仓库名称（从URL中提取）
repo_identifier = (
repo_url.split("/tree/")[0].split("/")[-1].replace(".git", "")
)
# 获取文件列表
search_type = "RECURSIVE" if recursive else "DIRECT"
tree_list = await AliyunFileInfo.get_repository_tree(
repo_identifier, dir_path, branch, search_type
)
result = []
for tree in tree_list:
# 跳过非当前目录的文件（如果不是递归模式）
if (
not recursive
and tree.path != dir_path
and "/" in tree.path.replace(dir_path, "", 1).strip("/")
):
continue
file_info = RepoFileInfo(
path=tree.path,
is_dir=tree.type == "tree",
)
result.append(file_info)
return result
except Exception as e:
logger.error(f"获取文件列表失败: {e}")
return []
async def get_commit_info(
self, repo_url: str, commit_id: str
) -> RepoCommitInfo | None:
"""
获取提交信息
Args:
repo_url: 仓库URL或名称
commit_id: 提交ID
Returns:
Optional[RepoCommitInfo]: 提交信息，如果获取失败则返回None
"""
try:
# 检查配置
self._check_config()
# 获取仓库名称（从URL中提取）
repo_identifier = (
repo_url.split("/tree/")[0].split("/")[-1].replace(".git", "")
)
# 获取提交信息
# 注意：这里假设AliyunFileInfo有get_commit_info方法，如果没有，需要实现
commit_data = await self._get_commit_info(repo_identifier, commit_id)
if not commit_data:
return None
# 解析提交信息
id_value = commit_data.get("id", commit_id)
message_value = commit_data.get("message", "")
author_value = commit_data.get("author_name", "")
date_value = commit_data.get(
"authored_date", datetime.now().isoformat()
).replace("Z", "+00:00")
return RepoCommitInfo(
commit_id=id_value,
message=message_value,
author=author_value,
commit_time=datetime.fromisoformat(date_value),
changed_files=[], # 阿里云API可能没有直接提供变更文件列表
)
except Exception as e:
logger.error(f"获取提交信息失败: {e}")
return None
def _check_config(self):
"""检查配置"""
if not self.config.aliyun_codeup.access_key_id:
raise AuthenticationError("阿里云CodeUp")
if not self.config.aliyun_codeup.access_key_secret:
raise AuthenticationError("阿里云CodeUp")
if not self.config.aliyun_codeup.organization_id:
raise AuthenticationError("阿里云CodeUp")
async def _get_newest_commit(self, repo_name: str, branch: str) -> str:
"""
获取仓库最新提交ID
Args:
repo_name: 仓库名称
branch: 分支名称
Returns:
str: 提交ID
"""
try:
newest_commit = await AliyunFileInfo.get_newest_commit(repo_name, branch)
if not newest_commit:
raise RepoNotFoundError(repo_name)
return newest_commit
except Exception as e:
logger.error(f"获取最新提交ID失败: {e}")
raise RepoUpdateError(f"获取最新提交ID失败: {e}")
async def _get_commit_info(self, repo_name: str, commit_id: str) -> dict:
"""
获取提交信息
Args:
repo_name: 仓库名称
commit_id: 提交ID
Returns:
dict: 提交信息
"""
# 这里需要实现从阿里云获取提交信息的逻辑
# 由于AliyunFileInfo可能没有get_commit_info方法，这里提供一个简单的实现
try:
# 这里应该是调用阿里云API获取提交信息
# 这里只是一个示例，实际上需要根据阿里云API实现
return {
"id": commit_id,
"message": "提交信息",
"author_name": "作者",
"authored_date": datetime.now().isoformat(),
}
except Exception as e:
logger.error(f"获取提交信息失败: {e}")
return {}
@cached(ttl=3600)
async def _get_changed_files(
self, repo_name: str, old_commit: str | None, new_commit: str
) -> list[str]:
"""
获取两个提交之间变更的文件列表
Args:
repo_name: 仓库名称
old_commit: 旧提交ID，如果为None则获取所有文件
new_commit: 新提交ID
Returns:
list[str]: 变更的文件列表
"""
if not old_commit:
# 如果没有旧提交，则获取仓库中的所有文件
tree_list = await AliyunFileInfo.get_repository_tree(
repo_name, "", new_commit, "RECURSIVE"
)
return [tree.path for tree in tree_list if tree.type == "blob"]
# 获取两个提交之间的差异
try:
return []
except Exception as e:
logger.error(f"获取提交差异失败: {e}")
raise RepoUpdateError(f"获取提交差异失败: {e}")
async def update_via_git(
self,
repo_url: str,
local_path: Path,
branch: str = "main",
force: bool = False,
*,
repo_type: RepoType | None = None,
owner: str | None = None,
prepare_repo_url: Callable[[str], str] | None = None,
) -> RepoUpdateResult:
"""
通过Git命令直接更新仓库
参数:
repo_url: 仓库名称
local_path: 本地仓库路径
branch: 分支名称
force: 是否强制拉取
返回:
RepoUpdateResult: 更新结果
"""
# 定义预处理函数，构建阿里云CodeUp的URL
def prepare_aliyun_url(repo_url: str) -> str:
import base64
repo_name = repo_url.split("/tree/")[0].split("/")[-1].replace(".git", "")
# 构建仓库URL
# 阿里云CodeUp的仓库URL格式通常为：
# https://codeup.aliyun.com/{organization_id}/{organization_name}/{repo_name}.git
url = f"https://codeup.aliyun.com/{self.config.aliyun_codeup.organization_id}/{self.config.aliyun_codeup.organization_name}/{repo_name}.git"
# 添加访问令牌 - 使用base64解码后的令牌
if self.config.aliyun_codeup.rdc_access_token_encrypted:
try:
# 解码RDC访问令牌
token = base64.b64decode(
self.config.aliyun_codeup.rdc_access_token_encrypted.encode()
).decode()
# 阿里云CodeUp使用oauth2:token的格式进行身份验证
url = url.replace("https://", f"https://oauth2:{token}@")
logger.debug(f"使用RDC令牌构建阿里云URL: {url.split('@')[0]}@***")
except Exception as e:
logger.error(f"解码RDC令牌失败: {e}")
return url
# 调用基类的update_via_git方法
return await super().update_via_git(
repo_url=repo_url,
local_path=local_path,
branch=branch,
force=force,
repo_type=RepoType.ALIYUN,
owner=self.config.aliyun_codeup.organization_id,
prepare_repo_url=prepare_aliyun_url,
)
async def update(
self,
repo_url: str,
local_path: Path,
branch: str = "main",
use_git: bool = True,
force: bool = False,
include_patterns: list[str] | None = None,
exclude_patterns: list[str] | None = None,
) -> RepoUpdateResult:
"""
更新仓库，可选择使用Git命令或API方式
参数:
repo_url: 仓库名称
local_path: 本地保存路径
branch: 分支名称
use_git: 是否使用Git命令更新
include_patterns: 包含的文件模式列表，如 ["*.py", "docs/*.md"]
exclude_patterns: 排除的文件模式列表，如 ["__pycache__/*", "*.pyc"]
返回:
RepoUpdateResult: 更新结果
"""
if use_git:
return await self.update_via_git(repo_url, local_path, branch, force)
else:
return await self.update_repo(
repo_url, local_path, branch, include_patterns, exclude_patterns
)
async def _download_file(
self, repo_name: str, file_path: str, local_path: Path, ref: str
) -> int:
"""
下载文件
Args:
repo_name: 仓库名称
file_path: 文件在仓库中的路径
local_path: 本地保存路径
ref: 分支/标签/提交ID
Returns:
int: 文件大小（字节）
"""
# 确保目录存在
local_path.parent.mkdir(parents=True, exist_ok=True)
# 获取文件内容
for retry in range(self.config.aliyun_codeup.download_retry + 1):
try:
content = await AliyunFileInfo.get_file_content(
file_path, repo_name, ref
)
if content is None:
raise FileNotFoundError(file_path, repo_name)
# 保存文件
return await self.save_file_content(content.encode("utf-8"), local_path)
except FileNotFoundError as e:
# 这些错误不需要重试
raise e
except Exception as e:
if retry < self.config.aliyun_codeup.download_retry:
logger.warning("下载文件失败，将重试", LOG_COMMAND, e=e)
await asyncio.sleep(1)
continue
raise RepoDownloadError(f"下载文件失败: {e}")
raise RepoDownloadError("下载文件失败: 超过最大重试次数")

432
zhenxun/utils/repo_utils/base_manager.py Normal file
View File

@ -0,0 +1,432 @@
"""
仓库管理工具的基础管理器
"""
from abc import ABC, abstractmethod
from pathlib import Path
import aiofiles
from zhenxun.services.log import logger
from .config import LOG_COMMAND, RepoConfig
from .models import (
FileDownloadResult,
RepoCommitInfo,
RepoFileInfo,
RepoType,
RepoUpdateResult,
)
from .utils import check_git, filter_files, run_git_command
class BaseRepoManager(ABC):
"""仓库管理工具基础类"""
def __init__(self, config: RepoConfig | None = None):
"""
初始化仓库管理工具
参数:
config: 配置，如果为None则使用默认配置
"""
self.config = config or RepoConfig.get_instance()
self.config.ensure_dirs()
@abstractmethod
async def update_repo(
self,
repo_url: str,
local_path: Path,
branch: str = "main",
include_patterns: list[str] | None = None,
exclude_patterns: list[str] | None = None,
) -> RepoUpdateResult:
"""
更新仓库
参数:
repo_url: 仓库URL或名称
local_path: 本地保存路径
branch: 分支名称
include_patterns: 包含的文件模式列表，如 ["*.py", "docs/*.md"]
exclude_patterns: 排除的文件模式列表，如 ["__pycache__/*", "*.pyc"]
返回:
RepoUpdateResult: 更新结果
"""
pass
@abstractmethod
async def download_file(
self,
repo_url: str,
file_path: str,
local_path: Path,
branch: str = "main",
) -> FileDownloadResult:
"""
下载单个文件
参数:
repo_url: 仓库URL或名称
file_path: 文件在仓库中的路径
local_path: 本地保存路径
branch: 分支名称
返回:
FileDownloadResult: 下载结果
"""
pass
@abstractmethod
async def get_file_list(
self,
repo_url: str,
dir_path: str = "",
branch: str = "main",
recursive: bool = False,
) -> list[RepoFileInfo]:
"""
获取仓库文件列表
参数:
repo_url: 仓库URL或名称
dir_path: 目录路径，空字符串表示仓库根目录
branch: 分支名称
recursive: 是否递归获取子目录
返回:
List[RepoFileInfo]: 文件信息列表
"""
pass
@abstractmethod
async def get_commit_info(
self, repo_url: str, commit_id: str
) -> RepoCommitInfo | None:
"""
获取提交信息
参数:
repo_url: 仓库URL或名称
commit_id: 提交ID
返回:
Optional[RepoCommitInfo]: 提交信息，如果获取失败则返回None
"""
pass
async def save_file_content(self, content: bytes, local_path: Path) -> int:
"""
保存文件内容
参数:
content: 文件内容
local_path: 本地保存路径
返回:
int: 文件大小（字节）
"""
# 确保目录存在
local_path.parent.mkdir(parents=True, exist_ok=True)
# 保存文件
async with aiofiles.open(local_path, "wb") as f:
await f.write(content)
return len(content)
async def read_version_file(self, local_dir: Path) -> str:
"""
读取版本文件
参数:
local_dir: 本地目录
返回:
str: 版本号
"""
version_file = local_dir / "__version__"
if not version_file.exists():
return ""
try:
async with aiofiles.open(version_file) as f:
return (await f.read()).strip()
except Exception as e:
logger.error(f"读取版本文件失败: {e}")
return ""
async def write_version_file(self, local_dir: Path, version: str) -> bool:
"""
写入版本文件
参数:
local_dir: 本地目录
version: 版本号
返回:
bool: 是否成功
"""
version_file = local_dir / "__version__"
try:
version_bb = "vNone"
async with aiofiles.open(version_file) as rf:
if text := await rf.read():
version_bb = text.strip().split("-")[0]
async with aiofiles.open(version_file, "w") as f:
await f.write(f"{version_bb}-{version[:6]}")
return True
except Exception as e:
logger.error(f"写入版本文件失败: {e}")
return False
def filter_files(
self,
files: list[str],
include_patterns: list[str] | None = None,
exclude_patterns: list[str] | None = None,
) -> list[str]:
"""
过滤文件列表
参数:
files: 文件列表
include_patterns: 包含的文件模式列表，如 ["*.py", "docs/*.md"]
exclude_patterns: 排除的文件模式列表，如 ["__pycache__/*", "*.pyc"]
返回:
List[str]: 过滤后的文件列表
"""
return filter_files(files, include_patterns, exclude_patterns)
async def update_via_git(
self,
repo_url: str,
local_path: Path,
branch: str = "main",
force: bool = False,
*,
repo_type: RepoType | None = None,
owner="",
prepare_repo_url=None,
) -> RepoUpdateResult:
"""
通过Git命令直接更新仓库
参数:
repo_url: 仓库URL或名称
local_path: 本地仓库路径
branch: 分支名称
force: 是否强制拉取
repo_type: 仓库类型
owner: 仓库拥有者
prepare_repo_url: 预处理仓库URL的函数
返回:
RepoUpdateResult: 更新结果
"""
from .models import RepoType
repo_name = repo_url.split("/tree/")[0].split("/")[-1].replace(".git", "")
try:
# 创建结果对象
result = RepoUpdateResult(
repo_type=repo_type or RepoType.GITHUB, # 默认使用GitHub类型
repo_name=repo_name,
owner=owner or "",
old_version="",
new_version="",
)
# 检查Git是否可用
if not await check_git():
return RepoUpdateResult(
repo_type=repo_type or RepoType.GITHUB,
repo_name=repo_name,
owner=owner or "",
old_version="",
new_version="",
error_message="Git命令不可用",
)
# 预处理仓库URL
if prepare_repo_url:
repo_url = prepare_repo_url(repo_url)
# 检查本地目录是否存在
if not local_path.exists():
# 如果不存在，则克隆仓库
logger.info(f"克隆仓库 {repo_url} 到 {local_path}", LOG_COMMAND)
success, stdout, stderr = await run_git_command(
f"clone -b {branch} {repo_url} {local_path}"
)
if not success:
return RepoUpdateResult(
repo_type=repo_type or RepoType.GITHUB,
repo_name=repo_name,
owner=owner or "",
old_version="",
new_version="",
error_message=f"克隆仓库失败: {stderr}",
)
# 获取当前提交ID
success, new_version, _ = await run_git_command(
"rev-parse HEAD", cwd=local_path
)
result.new_version = new_version.strip()
result.success = True
return result
# 如果目录存在，检查是否是Git仓库
# 首先检查目录本身是否有.git文件夹
git_dir = local_path / ".git"
if not git_dir.is_dir():
# 如果不是Git仓库，尝试初始化它
logger.info(f"目录 {local_path} 不是Git仓库，尝试初始化", LOG_COMMAND)
init_success, _, init_stderr = await run_git_command(
"init", cwd=local_path
)
if not init_success:
return RepoUpdateResult(
repo_type=repo_type or RepoType.GITHUB,
repo_name=repo_name,
owner=owner or "",
old_version="",
new_version="",
error_message=f"初始化Git仓库失败: {init_stderr}",
)
# 添加远程仓库
remote_success, _, remote_stderr = await run_git_command(
f"remote add origin {repo_url}", cwd=local_path
)
if not remote_success:
return RepoUpdateResult(
repo_type=repo_type or RepoType.GITHUB,
repo_name=repo_name,
owner=owner or "",
old_version="",
new_version="",
error_message=f"添加远程仓库失败: {remote_stderr}",
)
logger.info(f"成功初始化Git仓库 {local_path}", LOG_COMMAND)
# 获取当前提交ID作为旧版本
success, old_version, _ = await run_git_command(
"rev-parse HEAD", cwd=local_path
)
result.old_version = old_version.strip()
# 获取当前远程URL
success, remote_url, _ = await run_git_command(
"config --get remote.origin.url", cwd=local_path
)
# 如果远程URL不匹配，则更新它
remote_url = remote_url.strip()
if success and repo_url not in remote_url and remote_url not in repo_url:
logger.info(f"更新远程URL: {remote_url} -> {repo_url}", LOG_COMMAND)
await run_git_command(
f"remote set-url origin {repo_url}", cwd=local_path
)
# 获取远程更新
logger.info(f"获取远程更新: {repo_url}", LOG_COMMAND)
success, _, stderr = await run_git_command("fetch origin", cwd=local_path)
if not success:
return RepoUpdateResult(
repo_type=repo_type or RepoType.GITHUB,
repo_name=repo_name,
owner=owner or "",
old_version=old_version.strip(),
new_version="",
error_message=f"获取远程更新失败: {stderr}",
)
# 获取当前分支
success, current_branch, _ = await run_git_command(
"rev-parse --abbrev-ref HEAD", cwd=local_path
)
current_branch = current_branch.strip()
# 如果当前分支不是目标分支，则切换分支
if success and current_branch != branch:
logger.info(f"切换分支: {current_branch} -> {branch}", LOG_COMMAND)
success, _, stderr = await run_git_command(
f"checkout {branch}", cwd=local_path
)
if not success:
return RepoUpdateResult(
repo_type=repo_type or RepoType.GITHUB,
repo_name=repo_name,
owner=owner or "",
old_version=old_version.strip(),
new_version="",
error_message=f"切换分支失败: {stderr}",
)
# 拉取最新代码
logger.info(f"拉取最新代码: {repo_url}", LOG_COMMAND)
pull_cmd = f"pull origin {branch}"
if force:
pull_cmd = f"fetch --all && git reset --hard origin/{branch}"
logger.info("使用强制拉取模式", LOG_COMMAND)
success, _, stderr = await run_git_command(pull_cmd, cwd=local_path)
if not success:
return RepoUpdateResult(
repo_type=repo_type or RepoType.GITHUB,
repo_name=repo_name,
owner=owner or "",
old_version=old_version.strip(),
new_version="",
error_message=f"拉取最新代码失败: {stderr}",
)
# 获取更新后的提交ID
success, new_version, _ = await run_git_command(
"rev-parse HEAD", cwd=local_path
)
result.new_version = new_version.strip()
# 如果版本相同，则无需更新
if old_version.strip() == new_version.strip():
logger.info(
f"仓库 {repo_url} 已是最新版本: {new_version.strip()}", LOG_COMMAND
)
result.success = True
return result
# 获取变更的文件列表
success, changed_files_output, _ = await run_git_command(
f"diff --name-only {old_version.strip()} {new_version.strip()}",
cwd=local_path,
)
if success:
changed_files = [
line.strip()
for line in changed_files_output.splitlines()
if line.strip()
]
result.changed_files = changed_files
logger.info(f"变更的文件列表: {changed_files}", LOG_COMMAND)
result.success = True
return result
except Exception as e:
logger.error("Git更新失败", LOG_COMMAND, e=e)
return RepoUpdateResult(
repo_type=repo_type or RepoType.GITHUB,
repo_name=repo_name,
owner=owner or "",
old_version="",
new_version="",
error_message=str(e),
)

77
zhenxun/utils/repo_utils/config.py Normal file
View File

@ -0,0 +1,77 @@
"""
仓库管理工具的配置模块
"""
from dataclasses import dataclass, field
from pathlib import Path
from zhenxun.configs.path_config import TEMP_PATH
LOG_COMMAND = "RepoUtils"
@dataclass
class GithubConfig:
"""GitHub配置"""
# API超时时间（秒）
api_timeout: int = 30
# 下载超时时间（秒）
download_timeout: int = 60
# 下载重试次数
download_retry: int = 3
# 代理配置
proxy: str | None = None
@dataclass
class AliyunCodeupConfig:
"""阿里云CodeUp配置"""
# 访问密钥ID
access_key_id: str = "LTAI5tNmf7KaTAuhcvRobAQs"
# 访问密钥密钥
access_key_secret: str = "NmJ3d2VNRU1MREY0T1RtRnBqMlFqdlBxN3pMUk1j"
# 组织ID
organization_id: str = "67a361cf556e6cdab537117a"
# 组织名称
organization_name: str = "zhenxun-org"
# RDC Access Token
rdc_access_token_encrypted: str = (
"cHQtYXp0allnQWpub0FYZWpqZm1RWGtneHk0XzBlMmYzZTZmLWQwOWItNDE4Mi1iZWUx"
"LTQ1ZTFkYjI0NGRlMg=="
)
# 区域
region: str = "cn-hangzhou"
# 端点
endpoint: str = "devops.cn-hangzhou.aliyuncs.com"
# 下载重试次数
download_retry: int = 3
@dataclass
class RepoConfig:
"""仓库管理工具配置"""
# 缓存目录
cache_dir: Path = TEMP_PATH / "repo_cache"
# GitHub配置
github: GithubConfig = field(default_factory=GithubConfig)
# 阿里云CodeUp配置
aliyun_codeup: AliyunCodeupConfig = field(default_factory=AliyunCodeupConfig)
# 单例实例
_instance = None
@classmethod
def get_instance(cls) -> "RepoConfig":
"""获取单例实例"""
if cls._instance is None:
cls._instance = cls()
return cls._instance
def ensure_dirs(self):
"""确保目录存在"""
self.cache_dir.mkdir(parents=True, exist_ok=True)

68
zhenxun/utils/repo_utils/exceptions.py Normal file
View File

@ -0,0 +1,68 @@
"""
仓库管理工具的异常类
"""
class RepoManagerError(Exception):
"""仓库管理工具异常基类"""
def __init__(self, message: str, repo_name: str | None = None):
self.message = message
self.repo_name = repo_name
super().__init__(self.message)
class RepoUpdateError(RepoManagerError):
"""仓库更新异常"""
def __init__(self, message: str, repo_name: str | None = None):
super().__init__(f"仓库更新失败: {message}", repo_name)
class RepoDownloadError(RepoManagerError):
"""仓库下载异常"""
def __init__(self, message: str, repo_name: str | None = None):
super().__init__(f"文件下载失败: {message}", repo_name)
class RepoNotFoundError(RepoManagerError):
"""仓库不存在异常"""
def __init__(self, repo_name: str):
super().__init__(f"仓库不存在: {repo_name}", repo_name)
class FileNotFoundError(RepoManagerError):
"""文件不存在异常"""
def __init__(self, file_path: str, repo_name: str | None = None):
super().__init__(f"文件不存在: {file_path}", repo_name)
class AuthenticationError(RepoManagerError):
"""认证异常"""
def __init__(self, repo_type: str):
super().__init__(f"认证失败: {repo_type}")
class ApiRateLimitError(RepoManagerError):
"""API速率限制异常"""
def __init__(self, repo_type: str):
super().__init__(f"API速率限制: {repo_type}")
class NetworkError(RepoManagerError):
"""网络异常"""
def __init__(self, message: str):
super().__init__(f"网络错误: {message}")
class ConfigError(RepoManagerError):
"""配置异常"""
def __init__(self, message: str):
super().__init__(f"配置错误: {message}")

543
zhenxun/utils/repo_utils/file_manager.py Normal file
View File

@ -0,0 +1,543 @@
"""
仓库文件管理器，用于从GitHub和阿里云CodeUp获取指定文件内容
"""
from pathlib import Path
from typing import cast, overload
import aiofiles
from httpx import Response
from zhenxun.services.log import logger
from zhenxun.utils.github_utils import GithubUtils
from zhenxun.utils.github_utils.models import AliyunTreeType, GitHubStrategy, TreeType
from zhenxun.utils.http_utils import AsyncHttpx
from .config import LOG_COMMAND, RepoConfig
from .exceptions import FileNotFoundError, NetworkError, RepoManagerError
from .models import FileDownloadResult, RepoFileInfo, RepoType
class RepoFileManager:
"""仓库文件管理器，用于获取GitHub和阿里云仓库中的文件内容"""
def __init__(self, config: RepoConfig | None = None):
"""
初始化仓库文件管理器
参数:
config: 配置，如果为None则使用默认配置
"""
self.config = config or RepoConfig.get_instance()
self.config.ensure_dirs()
@overload
async def get_github_file_content(
self, url: str, file_path: str, ignore_error: bool = False
) -> str: ...
@overload
async def get_github_file_content(
self, url: str, file_path: list[str], ignore_error: bool = False
) -> list[tuple[str, str]]: ...
async def get_github_file_content(
self, url: str, file_path: str | list[str], ignore_error: bool = False
) -> str | list[tuple[str, str]]:
"""
获取GitHub仓库文件内容
参数:
url: 仓库URL
file_path: 文件路径或文件路径列表
ignore_error: 是否忽略错误
返回:
list[tuple[str, str]]: 文件路径，文件内容
"""
results = []
is_str_input = isinstance(file_path, str)
try:
if is_str_input:
file_path = [file_path]
repo_info = GithubUtils.parse_github_url(url)
if await repo_info.update_repo_commit():
logger.info(f"获取最新提交: {repo_info.branch}", LOG_COMMAND)
else:
logger.warning(f"获取最新提交失败: {repo_info}", LOG_COMMAND)
for f in file_path:
try:
file_url = await repo_info.get_raw_download_urls(f)
for fu in file_url:
response: Response = await AsyncHttpx.get(
fu, check_status_code=200
)
if response.status_code == 200:
logger.info(f"获取github文件内容成功: {f}", LOG_COMMAND)
# 确保使用UTF-8编码解析响应内容
try:
text_content = response.content.decode("utf-8")
except UnicodeDecodeError:
# 如果UTF-8解码失败，尝试其他编码
text_content = response.content.decode(
"utf-8", errors="ignore"
)
logger.warning(
f"解码文件内容时出现错误，使用忽略错误模式: {f}",
LOG_COMMAND,
)
results.append((f, text_content))
break
else:
logger.warning(
f"获取github文件内容失败: {response.status_code}",
LOG_COMMAND,
)
except Exception as e:
logger.warning(f"获取github文件内容失败: {f}", LOG_COMMAND, e=e)
if not ignore_error:
raise
except Exception as e:
logger.error(f"获取GitHub文件内容失败: {file_path}", LOG_COMMAND, e=e)
raise
logger.debug(f"获取GitHub文件内容: {[r[0] for r in results]}", LOG_COMMAND)
return results[0][1] if is_str_input and results else results
@overload
async def get_aliyun_file_content(
self,
repo_name: str,
file_path: str,
branch: str = "main",
ignore_error: bool = False,
) -> str: ...
@overload
async def get_aliyun_file_content(
self,
repo_name: str,
file_path: list[str],
branch: str = "main",
ignore_error: bool = False,
) -> list[tuple[str, str]]: ...
async def get_aliyun_file_content(
self,
repo_name: str,
file_path: str | list[str],
branch: str = "main",
ignore_error: bool = False,
) -> str | list[tuple[str, str]]:
"""
获取阿里云CodeUp仓库文件内容
参数:
repo: 仓库名称
file_path: 文件路径
branch: 分支名称
ignore_error: 是否忽略错误
返回:
list[tuple[str, str]]: 文件路径，文件内容
"""
results = []
is_str_input = isinstance(file_path, str)
# 导入阿里云相关模块
from zhenxun.utils.github_utils.models import AliyunFileInfo
if is_str_input:
file_path = [file_path]
for f in file_path:
try:
content = await AliyunFileInfo.get_file_content(
file_path=f, repo=repo_name, ref=branch
)
results.append((f, content))
except Exception as e:
logger.warning(f"获取阿里云文件内容失败: {file_path}", LOG_COMMAND, e=e)
if not ignore_error:
raise
logger.debug(f"获取阿里云文件内容: {[r[0] for r in results]}", LOG_COMMAND)
return results[0][1] if is_str_input and results else results
@overload
async def get_file_content(
self,
repo_url: str,
file_path: str,
branch: str = "main",
repo_type: RepoType | None = None,
ignore_error: bool = False,
) -> str: ...
@overload
async def get_file_content(
self,
repo_url: str,
file_path: list[str],
branch: str = "main",
repo_type: RepoType | None = None,
ignore_error: bool = False,
) -> list[tuple[str, str]]: ...
async def get_file_content(
self,
repo_url: str,
file_path: str | list[str],
branch: str = "main",
repo_type: RepoType | None = None,
ignore_error: bool = False,
) -> str | list[tuple[str, str]]:
"""
获取仓库文件内容
参数:
repo_url: 仓库URL
file_path: 文件路径
branch: 分支名称
repo_type: 仓库类型，如果为None则自动判断
ignore_error: 是否忽略错误
返回:
str: 文件内容
"""
# 确定仓库类型
repo_name = (
repo_url.split("/tree/")[0].split("/")[-1].replace(".git", "").strip()
)
if repo_type is None:
try:
return await self.get_aliyun_file_content(
repo_name, file_path, branch, ignore_error
)
except Exception:
return await self.get_github_file_content(
repo_url, file_path, ignore_error
)
try:
if repo_type == RepoType.GITHUB:
return await self.get_github_file_content(
repo_url, file_path, ignore_error
)
elif repo_type == RepoType.ALIYUN:
return await self.get_aliyun_file_content(
repo_name, file_path, branch, ignore_error
)
except Exception as e:
if isinstance(e, FileNotFoundError | NetworkError | RepoManagerError):
raise
raise RepoManagerError(f"获取文件内容失败: {e}")
async def list_directory_files(
self,
repo_url: str,
directory_path: str = "",
branch: str = "main",
repo_type: RepoType | None = None,
recursive: bool = True,
) -> list[RepoFileInfo]:
"""
获取仓库目录下的所有文件路径
参数:
repo_url: 仓库URL
directory_path: 目录路径，默认为仓库根目录
branch: 分支名称
repo_type: 仓库类型，如果为None则自动判断
recursive: 是否递归获取子目录文件
返回:
list[RepoFileInfo]: 文件信息列表
"""
repo_name = (
repo_url.split("/tree/")[0].split("/")[-1].replace(".git", "").strip()
)
try:
if repo_type is None:
# 尝试GitHub，失败则尝试阿里云
try:
return await self._list_github_directory_files(
repo_url, directory_path, branch, recursive
)
except Exception as e:
logger.warning(
"获取GitHub目录文件失败，尝试阿里云", LOG_COMMAND, e=e
)
return await self._list_aliyun_directory_files(
repo_name, directory_path, branch, recursive
)
if repo_type == RepoType.GITHUB:
return await self._list_github_directory_files(
repo_url, directory_path, branch, recursive
)
elif repo_type == RepoType.ALIYUN:
return await self._list_aliyun_directory_files(
repo_name, directory_path, branch, recursive
)
except Exception as e:
logger.error(f"获取目录文件列表失败: {directory_path}", LOG_COMMAND, e=e)
if isinstance(e, FileNotFoundError | NetworkError | RepoManagerError):
raise
raise RepoManagerError(f"获取目录文件列表失败: {e}")
async def _list_github_directory_files(
self,
repo_url: str,
directory_path: str = "",
branch: str = "main",
recursive: bool = True,
build_tree: bool = False,
) -> list[RepoFileInfo]:
"""
获取GitHub仓库目录下的所有文件路径
参数:
repo_url: 仓库URL
directory_path: 目录路径，默认为仓库根目录
branch: 分支名称
recursive: 是否递归获取子目录文件
build_tree: 是否构建目录树
返回:
list[RepoFileInfo]: 文件信息列表
"""
try:
repo_info = GithubUtils.parse_github_url(repo_url)
if await repo_info.update_repo_commit():
logger.info(f"获取最新提交: {repo_info.branch}", LOG_COMMAND)
else:
logger.warning(f"获取最新提交失败: {repo_info}", LOG_COMMAND)
# 获取仓库树信息
strategy = GitHubStrategy()
strategy.body = await GitHubStrategy.parse_repo_info(repo_info)
# 处理目录路径，确保格式正确
if directory_path and not directory_path.endswith("/") and recursive:
directory_path = f"{directory_path}/"
# 获取文件列表
file_list = []
for tree_item in strategy.body.tree:
# 如果不是递归模式，只获取当前目录下的文件
if not recursive and "/" in tree_item.path.replace(
directory_path, "", 1
):
continue
# 检查是否在指定目录下
if directory_path and not tree_item.path.startswith(directory_path):
continue
# 创建文件信息对象
file_info = RepoFileInfo(
path=tree_item.path,
is_dir=tree_item.type == TreeType.DIR,
size=tree_item.size,
last_modified=None, # GitHub API不直接提供最后修改时间
)
file_list.append(file_info)
# 构建目录树结构
if recursive and build_tree:
file_list = self._build_directory_tree(file_list)
return file_list
except Exception as e:
logger.error(
f"获取GitHub目录文件列表失败: {directory_path}", LOG_COMMAND, e=e
)
raise
async def _list_aliyun_directory_files(
self,
repo_name: str,
directory_path: str = "",
branch: str = "main",
recursive: bool = True,
build_tree: bool = False,
) -> list[RepoFileInfo]:
"""
获取阿里云CodeUp仓库目录下的所有文件路径
参数:
repo_name: 仓库名称
directory_path: 目录路径，默认为仓库根目录
branch: 分支名称
recursive: 是否递归获取子目录文件
build_tree: 是否构建目录树
返回:
list[RepoFileInfo]: 文件信息列表
"""
try:
from zhenxun.utils.github_utils.models import AliyunFileInfo
# 获取仓库树信息
search_type = "RECURSIVE" if recursive else "DIRECT"
tree_list = await AliyunFileInfo.get_repository_tree(
repo=repo_name,
path=directory_path,
ref=branch,
search_type=search_type,
)
# 创建文件信息对象列表
file_list = []
for tree_item in tree_list:
file_info = RepoFileInfo(
path=tree_item.path,
is_dir=tree_item.type == AliyunTreeType.DIR,
size=None, # 阿里云API不直接提供文件大小
last_modified=None, # 阿里云API不直接提供最后修改时间
)
file_list.append(file_info)
# 构建目录树结构
if recursive and build_tree:
file_list = self._build_directory_tree(file_list)
return file_list
except Exception as e:
logger.error(
f"获取阿里云目录文件列表失败: {directory_path}", LOG_COMMAND, e=e
)
raise
def _build_directory_tree(
self, file_list: list[RepoFileInfo]
) -> list[RepoFileInfo]:
"""
构建目录树结构
参数:
file_list: 文件信息列表
返回:
list[RepoFileInfo]: 根目录下的文件信息列表
"""
# 按路径排序，确保父目录在子目录之前
file_list.sort(key=lambda x: x.path)
# 创建路径到文件信息的映射
path_map = {file_info.path: file_info for file_info in file_list}
# 根目录文件列表
root_files = []
for file_info in file_list:
if parent_path := "/".join(file_info.path.split("/")[:-1]):
# 如果有父目录，将当前文件添加到父目录的子文件列表中
if parent_path in path_map:
path_map[parent_path].children.append(file_info)
else:
# 如果父目录不在列表中，创建一个虚拟的父目录
parent_info = RepoFileInfo(
path=parent_path, is_dir=True, children=[file_info]
)
path_map[parent_path] = parent_info
# 检查父目录的父目录
grand_parent_path = "/".join(parent_path.split("/")[:-1])
if grand_parent_path and grand_parent_path in path_map:
path_map[grand_parent_path].children.append(parent_info)
else:
root_files.append(parent_info)
else:
# 如果没有父目录，则是根目录下的文件
root_files.append(file_info)
# 返回根目录下的文件列表
return [
file
for file in root_files
if all(f.path != file.path for f in file_list if f != file)
]
async def download_files(
self,
repo_url: str,
file_path: tuple[str, Path] | list[tuple[str, Path]],
branch: str = "main",
repo_type: RepoType | None = None,
ignore_error: bool = False,
) -> FileDownloadResult:
"""
下载单个文件
参数:
repo_url: 仓库URL
file_path: 文件在仓库中的路径，本地存储路径
branch: 分支名称
repo_type: 仓库类型，如果为None则自动判断
ignore_error: 是否忽略错误
返回:
FileDownloadResult: 下载结果
"""
# 确定仓库类型和所有者
repo_name = (
repo_url.split("/tree/")[0].split("/")[-1].replace(".git", "").strip()
)
if isinstance(file_path, tuple):
file_path = [file_path]
file_path_mapping = {f[0]: f[1] for f in file_path}
# 创建结果对象
result = FileDownloadResult(
repo_type=repo_type,
repo_name=repo_name,
file_path=file_path,
version=branch,
)
try:
# 由于我们传入的是列表，所以这里一定返回列表
file_paths = [f[0] for f in file_path]
if len(file_paths) == 1:
# 如果只有一个文件，可能返回单个元组
file_contents_result = await self.get_file_content(
repo_url, file_paths[0], branch, repo_type, ignore_error
)
if isinstance(file_contents_result, tuple):
file_contents = [file_contents_result]
elif isinstance(file_contents_result, str):
file_contents = [(file_paths[0], file_contents_result)]
else:
file_contents = cast(list[tuple[str, str]], file_contents_result)
else:
# 多个文件一定返回列表
file_contents = cast(
list[tuple[str, str]],
await self.get_file_content(
repo_url, file_paths, branch, repo_type, ignore_error
),
)
for repo_file_path, content in file_contents:
local_path = file_path_mapping[repo_file_path]
local_path.parent.mkdir(parents=True, exist_ok=True)
# 使用二进制模式写入文件，避免编码问题
if isinstance(content, str):
content_bytes = content.encode("utf-8")
else:
content_bytes = content
logger.warning(f"写入文件: {local_path}")
async with aiofiles.open(local_path, "wb") as f:
await f.write(content_bytes)
result.success = True
# 计算文件大小
result.file_size = sum(
len(content.encode("utf-8") if isinstance(content, str) else content)
for _, content in file_contents
)
return result
except Exception as e:
logger.error(f"下载文件失败: {e}")
result.success = False
result.error_message = str(e)
return result

526
zhenxun/utils/repo_utils/github_manager.py Normal file
View File

@ -0,0 +1,526 @@
"""
GitHub仓库管理工具
"""
import asyncio
from collections.abc import Callable
from datetime import datetime
from pathlib import Path
from aiocache import cached
from zhenxun.services.log import logger
from zhenxun.utils.github_utils import GithubUtils, RepoInfo
from zhenxun.utils.http_utils import AsyncHttpx
from .base_manager import BaseRepoManager
from .config import LOG_COMMAND, RepoConfig
from .exceptions import (
ApiRateLimitError,
FileNotFoundError,
NetworkError,
RepoDownloadError,
RepoNotFoundError,
RepoUpdateError,
)
from .models import (
FileDownloadResult,
RepoCommitInfo,
RepoFileInfo,
RepoType,
RepoUpdateResult,
)
class GithubManager(BaseRepoManager):
"""GitHub仓库管理工具"""
def __init__(self, config: RepoConfig | None = None):
"""
初始化GitHub仓库管理工具
参数:
config: 配置，如果为None则使用默认配置
"""
super().__init__(config)
async def update_repo(
self,
repo_url: str,
local_path: Path,
branch: str = "main",
include_patterns: list[str] | None = None,
exclude_patterns: list[str] | None = None,
) -> RepoUpdateResult:
"""
更新GitHub仓库
参数:
repo_url: 仓库URL，格式为 https://github.com/owner/repo
local_path: 本地保存路径
branch: 分支名称
include_patterns: 包含的文件模式列表，如 ["*.py", "docs/*.md"]
exclude_patterns: 排除的文件模式列表，如 ["__pycache__/*", "*.pyc"]
返回:
RepoUpdateResult: 更新结果
"""
try:
# 解析仓库URL
repo_info = GithubUtils.parse_github_url(repo_url)
repo_info.branch = branch
# 获取仓库最新提交ID
newest_commit = await self._get_newest_commit(
repo_info.owner, repo_info.repo, branch
)
# 创建结果对象
result = RepoUpdateResult(
repo_type=RepoType.GITHUB,
repo_name=repo_info.repo,
owner=repo_info.owner,
old_version="", # 将在后面更新
new_version=newest_commit,
)
old_version = await self.read_version_file(local_path)
old_version = old_version.split("-")[-1]
result.old_version = old_version
# 如果版本相同，则无需更新
if newest_commit in old_version:
result.success = True
logger.debug(
f"仓库 {repo_info.repo} 已是最新版本: {newest_commit}",
LOG_COMMAND,
)
return result
# 确保本地目录存在
local_path.mkdir(parents=True, exist_ok=True)
# 获取变更的文件列表
changed_files = await self._get_changed_files(
repo_info.owner,
repo_info.repo,
old_version or None,
newest_commit,
)
# 过滤文件
if include_patterns or exclude_patterns:
from .utils import filter_files
changed_files = filter_files(
changed_files, include_patterns, exclude_patterns
)
result.changed_files = changed_files
# 下载变更的文件
for file_path in changed_files:
try:
local_file_path = local_path / file_path
await self._download_file(repo_info, file_path, local_file_path)
except Exception as e:
logger.error(f"下载文件 {file_path} 失败", LOG_COMMAND, e=e)
# 更新版本文件
await self.write_version_file(local_path, newest_commit)
result.success = True
return result
except RepoUpdateError as e:
logger.error("更新仓库失败", LOG_COMMAND, e=e)
return RepoUpdateResult(
repo_type=RepoType.GITHUB,
repo_name=repo_url.split("/")[-1] if "/" in repo_url else repo_url,
owner=repo_url.split("/")[-2] if "/" in repo_url else "unknown",
old_version="",
new_version="",
error_message=str(e),
)
except Exception as e:
logger.error("更新仓库失败", LOG_COMMAND, e=e)
return RepoUpdateResult(
repo_type=RepoType.GITHUB,
repo_name=repo_url.split("/")[-1] if "/" in repo_url else repo_url,
owner=repo_url.split("/")[-2] if "/" in repo_url else "unknown",
old_version="",
new_version="",
error_message=str(e),
)
async def download_file(
self,
repo_url: str,
file_path: str,
local_path: Path,
branch: str = "main",
) -> FileDownloadResult:
"""
从GitHub下载单个文件
参数:
repo_url: 仓库URL，格式为 https://github.com/owner/repo
file_path: 文件在仓库中的路径
local_path: 本地保存路径
branch: 分支名称
返回:
FileDownloadResult: 下载结果
"""
repo_name = (
repo_url.split("/tree/")[0].split("/")[-1].replace(".git", "").strip()
)
try:
# 解析仓库URL
repo_info = GithubUtils.parse_github_url(repo_url)
repo_info.branch = branch
# 创建结果对象
result = FileDownloadResult(
repo_type=RepoType.GITHUB,
repo_name=repo_info.repo,
file_path=file_path,
version=branch,
)
# 确保本地目录存在
local_path.parent.mkdir(parents=True, exist_ok=True)
# 下载文件
file_size = await self._download_file(repo_info, file_path, local_path)
result.success = True
result.file_size = file_size
return result
except RepoDownloadError as e:
logger.error("下载文件失败", LOG_COMMAND, e=e)
return FileDownloadResult(
repo_type=RepoType.GITHUB,
repo_name=repo_name,
file_path=file_path,
version=branch,
error_message=str(e),
)
except Exception as e:
logger.error("下载文件失败", LOG_COMMAND, e=e)
return FileDownloadResult(
repo_type=RepoType.GITHUB,
repo_name=repo_name,
file_path=file_path,
version=branch,
error_message=str(e),
)
async def get_file_list(
self,
repo_url: str,
dir_path: str = "",
branch: str = "main",
recursive: bool = False,
) -> list[RepoFileInfo]:
"""
获取仓库文件列表
参数:
repo_url: 仓库URL，格式为 https://github.com/owner/repo
dir_path: 目录路径，空字符串表示仓库根目录
branch: 分支名称
recursive: 是否递归获取子目录
返回:
list[RepoFileInfo]: 文件信息列表
"""
try:
# 解析仓库URL
repo_info = GithubUtils.parse_github_url(repo_url)
repo_info.branch = branch
# 获取文件列表
for api in GithubUtils.iter_api_strategies():
try:
await api.parse_repo_info(repo_info)
files = api.get_files(dir_path, True)
result = []
for file_path in files:
# 跳过非当前目录的文件（如果不是递归模式）
if not recursive and "/" in file_path.replace(
dir_path, "", 1
).strip("/"):
continue
is_dir = file_path.endswith("/")
file_info = RepoFileInfo(path=file_path, is_dir=is_dir)
result.append(file_info)
return result
except Exception as e:
logger.debug("使用API策略获取文件列表失败", LOG_COMMAND, e=e)
continue
raise RepoNotFoundError(repo_url)
except Exception as e:
logger.error("获取文件列表失败", LOG_COMMAND, e=e)
return []
async def get_commit_info(
self, repo_url: str, commit_id: str
) -> RepoCommitInfo | None:
"""
获取提交信息
参数:
repo_url: 仓库URL，格式为 https://github.com/owner/repo
commit_id: 提交ID
返回:
Optional[RepoCommitInfo]: 提交信息，如果获取失败则返回None
"""
try:
# 解析仓库URL
repo_info = GithubUtils.parse_github_url(repo_url)
# 构建API URL
api_url = f"https://api.github.com/repos/{repo_info.owner}/{repo_info.repo}/commits/{commit_id}"
# 发送请求
resp = await AsyncHttpx.get(
api_url,
timeout=self.config.github.api_timeout,
proxy=self.config.github.proxy,
)
if resp.status_code == 403 and "rate limit" in resp.text.lower():
raise ApiRateLimitError("GitHub")
if resp.status_code != 200:
if resp.status_code == 404:
raise RepoNotFoundError(f"{repo_info.owner}/{repo_info.repo}")
raise NetworkError(f"HTTP {resp.status_code}: {resp.text}")
data = resp.json()
return RepoCommitInfo(
commit_id=data["sha"],
message=data["commit"]["message"],
author=data["commit"]["author"]["name"],
commit_time=datetime.fromisoformat(
data["commit"]["author"]["date"].replace("Z", "+00:00")
),
changed_files=[file["filename"] for file in data.get("files", [])],
)
except Exception as e:
logger.error("获取提交信息失败", LOG_COMMAND, e=e)
return None
async def _get_newest_commit(self, owner: str, repo: str, branch: str) -> str:
"""
获取仓库最新提交ID
参数:
owner: 仓库拥有者
repo: 仓库名称
branch: 分支名称
返回:
str: 提交ID
"""
try:
newest_commit = await RepoInfo.get_newest_commit(owner, repo, branch)
if not newest_commit:
raise RepoNotFoundError(f"{owner}/{repo}")
return newest_commit
except Exception as e:
logger.error("获取最新提交ID失败", LOG_COMMAND, e=e)
raise RepoUpdateError(f"获取最新提交ID失败: {e}")
@cached(ttl=3600)
async def _get_changed_files(
self, owner: str, repo: str, old_commit: str | None, new_commit: str
) -> list[str]:
"""
获取两个提交之间变更的文件列表
参数:
owner: 仓库拥有者
repo: 仓库名称
old_commit: 旧提交ID，如果为None则获取所有文件
new_commit: 新提交ID
返回:
list[str]: 变更的文件列表
"""
if not old_commit:
# 如果没有旧提交，则获取仓库中的所有文件
api_url = f"https://api.github.com/repos/{owner}/{repo}/git/trees/{new_commit}?recursive=1"
resp = await AsyncHttpx.get(
api_url,
timeout=self.config.github.api_timeout,
proxy=self.config.github.proxy,
)
if resp.status_code == 403 and "rate limit" in resp.text.lower():
raise ApiRateLimitError("GitHub")
if resp.status_code != 200:
if resp.status_code == 404:
raise RepoNotFoundError(f"{owner}/{repo}")
raise NetworkError(f"HTTP {resp.status_code}: {resp.text}")
data = resp.json()
return [
item["path"] for item in data.get("tree", []) if item["type"] == "blob"
]
# 如果有旧提交，则获取两个提交之间的差异
api_url = f"https://api.github.com/repos/{owner}/{repo}/compare/{old_commit}...{new_commit}"
resp = await AsyncHttpx.get(
api_url,
timeout=self.config.github.api_timeout,
proxy=self.config.github.proxy,
)
if resp.status_code == 403 and "rate limit" in resp.text.lower():
raise ApiRateLimitError("GitHub")
if resp.status_code != 200:
if resp.status_code == 404:
raise RepoNotFoundError(f"{owner}/{repo}")
raise NetworkError(f"HTTP {resp.status_code}: {resp.text}")
data = resp.json()
return [file["filename"] for file in data.get("files", [])]
async def update_via_git(
self,
repo_url: str,
local_path: Path,
branch: str = "main",
force: bool = False,
*,
repo_type: RepoType | None = None,
owner: str | None = None,
prepare_repo_url: Callable[[str], str] | None = None,
) -> RepoUpdateResult:
"""
通过Git命令直接更新仓库
参数:
repo_url: 仓库URL，格式为 https://github.com/owner/repo
local_path: 本地仓库路径
branch: 分支名称
force: 是否强制拉取
返回:
RepoUpdateResult: 更新结果
"""
# 解析仓库URL
repo_info = GithubUtils.parse_github_url(repo_url)
# 调用基类的update_via_git方法
return await super().update_via_git(
repo_url=repo_url,
local_path=local_path,
branch=branch,
force=force,
repo_type=RepoType.GITHUB,
owner=repo_info.owner,
)
async def update(
self,
repo_url: str,
local_path: Path,
branch: str = "main",
use_git: bool = True,
force: bool = False,
include_patterns: list[str] | None = None,
exclude_patterns: list[str] | None = None,
) -> RepoUpdateResult:
"""
更新仓库，可选择使用Git命令或API方式
参数:
repo_url: 仓库URL，格式为 https://github.com/owner/repo
local_path: 本地保存路径
branch: 分支名称
use_git: 是否使用Git命令更新
include_patterns: 包含的文件模式列表，如 ["*.py", "docs/*.md"]
exclude_patterns: 排除的文件模式列表，如 ["__pycache__/*", "*.pyc"]
返回:
RepoUpdateResult: 更新结果
"""
if use_git:
return await self.update_via_git(repo_url, local_path, branch, force)
else:
return await self.update_repo(
repo_url, local_path, branch, include_patterns, exclude_patterns
)
async def _download_file(
self, repo_info: RepoInfo, file_path: str, local_path: Path
) -> int:
"""
下载文件
参数:
repo_info: 仓库信息
file_path: 文件在仓库中的路径
local_path: 本地保存路径
返回:
int: 文件大小（字节）
"""
# 确保目录存在
local_path.parent.mkdir(parents=True, exist_ok=True)
# 获取下载URL
download_url = await repo_info.get_raw_download_url(file_path)
# 下载文件
for retry in range(self.config.github.download_retry + 1):
try:
resp = await AsyncHttpx.get(
download_url,
timeout=self.config.github.download_timeout,
)
if resp.status_code == 403 and "rate limit" in resp.text.lower():
raise ApiRateLimitError("GitHub")
if resp.status_code != 200:
if resp.status_code == 404:
raise FileNotFoundError(
file_path, f"{repo_info.owner}/{repo_info.repo}"
)
if retry < self.config.github.download_retry:
await asyncio.sleep(1)
continue
raise NetworkError(f"HTTP {resp.status_code}: {resp.text}")
# 保存文件
return await self.save_file_content(resp.content, local_path)
except (ApiRateLimitError, FileNotFoundError) as e:
# 这些错误不需要重试
raise e
except Exception as e:
if retry < self.config.github.download_retry:
logger.warning("下载文件失败，将重试", LOG_COMMAND, e=e)
await asyncio.sleep(1)
continue
raise RepoDownloadError("下载文件失败")
raise RepoDownloadError("下载文件失败: 超过最大重试次数")

89
zhenxun/utils/repo_utils/models.py Normal file
View File

@ -0,0 +1,89 @@
"""
仓库管理工具的数据模型
"""
from dataclasses import dataclass, field
from datetime import datetime
from enum import Enum
from pathlib import Path
class RepoType(str, Enum):
"""仓库类型"""
GITHUB = "github"
ALIYUN = "aliyun"
@dataclass
class RepoFileInfo:
"""仓库文件信息"""
# 文件路径
path: str
# 是否是目录
is_dir: bool
# 文件大小（字节）
size: int | None = None
# 最后修改时间
last_modified: datetime | None = None
# 子文件列表
children: list["RepoFileInfo"] = field(default_factory=list)
@dataclass
class RepoCommitInfo:
"""仓库提交信息"""
# 提交ID
commit_id: str
# 提交消息
message: str
# 作者
author: str
# 提交时间
commit_time: datetime
# 变更的文件列表
changed_files: list[str] = field(default_factory=list)
@dataclass
class RepoUpdateResult:
"""仓库更新结果"""
# 仓库类型
repo_type: RepoType
# 仓库名称
repo_name: str
# 仓库拥有者
owner: str
# 旧版本
old_version: str
# 新版本
new_version: str
# 是否成功
success: bool = False
# 错误消息
error_message: str = ""
# 变更的文件列表
changed_files: list[str] = field(default_factory=list)
@dataclass
class FileDownloadResult:
"""文件下载结果"""
# 仓库类型
repo_type: RepoType | None
# 仓库名称
repo_name: str
# 文件路径
file_path: list[tuple[str, Path]] | str
# 版本
version: str
# 是否成功
success: bool = False
# 文件大小（字节）
file_size: int = 0
# 错误消息
error_message: str = ""

135
zhenxun/utils/repo_utils/utils.py Normal file
View File

@ -0,0 +1,135 @@
"""
仓库管理工具的工具函数
"""
import asyncio
from pathlib import Path
import re
from zhenxun.services.log import logger
from .config import LOG_COMMAND
async def check_git() -> bool:
"""
检查环境变量中是否存在 git
返回:
bool: 是否存在git命令
"""
try:
process = await asyncio.create_subprocess_shell(
"git --version",
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
stdout, _ = await process.communicate()
return bool(stdout)
except Exception as e:
logger.error("检查git命令失败", LOG_COMMAND, e=e)
return False
async def clean_git(cwd: Path):
"""
清理git仓库
参数:
cwd: 工作目录
"""
await run_git_command("reset --hard", cwd)
await run_git_command("clean -xdf", cwd)
async def run_git_command(
command: str, cwd: Path | None = None
) -> tuple[bool, str, str]:
"""
运行git命令
参数:
command: 命令
cwd: 工作目录
返回:
tuple[bool, str, str]: (是否成功, 标准输出, 标准错误)
"""
try:
full_command = f"git {command}"
# 将Path对象转换为字符串
cwd_str = str(cwd) if cwd else None
process = await asyncio.create_subprocess_shell(
full_command,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
cwd=cwd_str,
)
stdout_bytes, stderr_bytes = await process.communicate()
stdout = stdout_bytes.decode("utf-8").strip()
stderr = stderr_bytes.decode("utf-8").strip()
return process.returncode == 0, stdout, stderr
except Exception as e:
logger.error(f"运行git命令失败: {command}, 错误: {e}")
return False, "", str(e)
def glob_to_regex(pattern: str) -> str:
"""
将glob模式转换为正则表达式
参数:
pattern: glob模式，如 "*.py"
返回:
str: 正则表达式
"""
# 转义特殊字符
regex = re.escape(pattern)
# 替换glob通配符
regex = regex.replace(r"\*\*", ".*") # ** -> .*
regex = regex.replace(r"\*", "[^/]*") # * -> [^/]*
regex = regex.replace(r"\?", "[^/]") # ? -> [^/]
# 添加开始和结束标记
regex = f"^{regex}$"
return regex
def filter_files(
files: list[str],
include_patterns: list[str] | None = None,
exclude_patterns: list[str] | None = None,
) -> list[str]:
"""
过滤文件列表
参数:
files: 文件列表
include_patterns: 包含的文件模式列表，如 ["*.py", "docs/*.md"]
exclude_patterns: 排除的文件模式列表，如 ["__pycache__/*", "*.pyc"]
返回:
list[str]: 过滤后的文件列表
"""
result = files.copy()
# 应用包含模式
if include_patterns:
included = []
for pattern in include_patterns:
regex_pattern = glob_to_regex(pattern)
included.extend(file for file in result if re.match(regex_pattern, file))
result = included
# 应用排除模式
if exclude_patterns:
for pattern in exclude_patterns:
regex_pattern = glob_to_regex(pattern)
result = [file for file in result if not re.match(regex_pattern, file)]
return result