mirror of
https://github.com/Shu-Ying/main_plugin_farm.git
synced 2026-10-28 05:00:01 +08:00
fixed # 将资源文件迁移到Service仓库中
This commit is contained in:
@@ -29,7 +29,8 @@ wheel随包携带JSON及小型升级土地素材,不包含legacy、玩家数
|
||||
|
||||
- 注册500初始币、6地块;可靠识别为初始3块的旧号一次性补3块,
|
||||
保留已开垦成果。详见land-rules.md;价格/产量/成熟时间沿用目录。
|
||||
- 旧作物库202条记录完整导出且保留源 SHA-256。
|
||||
- 初始作物库202条记录完整迁入Service;当前数据发布和启动更新见
|
||||
[作物数据维护](catalogue-updates.md),旧库基线独立保留。
|
||||
- 多季再生长间隔明确保留去重 phase[1:] 求和,未重平衡数值。
|
||||
- F01/F02:余额和经验只增加本次增量。
|
||||
- F03:每块地单独加成、向下取整,再求和。
|
||||
|
||||
@@ -109,9 +109,10 @@ RenderService默认一个绘图任务,最多8个活跃/排队的不同请求
|
||||
|
||||
## 素材完整性与来源
|
||||
|
||||
scripts/export_render_assets.py将已有素材原样打包,附202种作物
|
||||
展示配置、旧布局和统一定位规则;额外打包新提取的 40 张网页素材。
|
||||
--check比对确定性输出与原图哈希,不修改旧基线。
|
||||
现行素材及展示配置由Service的catalogue/source维护,作为完整版本发布,
|
||||
插件启动时统一选择;见[作物数据维护](catalogue-updates.md)。
|
||||
scripts/export_render_assets.py仅保留显式旧基线导出;默认--check核对
|
||||
当前bootstrap,不再把日常配置绑定到旧库。
|
||||
素材包约束为安装资源,不按用户输入路径访问任意文件、不解压到磁盘。
|
||||
保留GPL及原始归属;历史素材分发权仍需在公开发布前核实。
|
||||
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
|
||||
| 文档 | 用途 |
|
||||
|---|---|
|
||||
| [catalogue-updates.md](catalogue-updates.md) | Service 作物维护、发布、启动更新、缓存与兼容回滚 |
|
||||
| [23-seasonal-events-and-rust-service.md](23-seasonal-events-and-rust-service.md) | 节日活动交互备选、Rust 发布中心、同步、资产事务、过期清理与上线规划(草案) |
|
||||
| [09-foundation.md](09-foundation.md) | 已实现的第一批核心、API、测试与限制 |
|
||||
| [10-rendering.md](10-rendering.md) | 共享图片工具、农场绘制、缓存、编码与接入 |
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
# 作物数据维护与启动更新
|
||||
|
||||
作物的可编辑源已迁入独立 Service 仓库的 `catalogue/source/`。
|
||||
插件内只保留首次离线启动所需的轻量基线;日常改价、调整生长参数和新增作物
|
||||
不需要重新发布插件。图片、展示配置、作物定义及升级土地图作为同一版本发布。
|
||||
|
||||
## 日常更新
|
||||
|
||||
1. 在 Service 修改 `catalogue/source/plants.json`。新增作物使用未占用的正整数
|
||||
ID,在 `catalogue/source/images/` 添加对应图片;布局在 `metadata.json`。
|
||||
2. 在 Service 根目录生成一个新的版本标签:
|
||||
|
||||
```text
|
||||
python scripts/prepare_catalogue.py --source catalogue/source --output catalogue --version crops-20261026-01
|
||||
python scripts/prepare_catalogue.py --source catalogue/source --output catalogue --version crops-20261026-01 --check
|
||||
```
|
||||
|
||||
3. 审查并发布 Service 中的源文件、`catalogue/manifest.json` 和新生成的
|
||||
`catalogue/releases/` 历史版本。Github/Gitea 源需将文件发布到相应仓库;
|
||||
HTTP 源需配置 `FARM_CATALOGUE_DIR` 指向完整 `catalogue`,并重启 Service
|
||||
以加载新快照。详细操作见 Service 的 `docs/catalogue.md`。
|
||||
4. 重启或重载农场插件。三框架共用启动流程,按配置的资源源检查一次更新;
|
||||
运行期间不轮询,不改变当前进程已加载的版本。改用 HTTP 源时使用有效的
|
||||
HTTPS Service 基础地址。
|
||||
|
||||
不可删除或改名已发布的作物 ID,也不可删除既有图片路径。停售或停止播种应
|
||||
调整对应开关。同一版本标签不可复用于不同内容。回滚应从旧配置生成新的兼容
|
||||
版本,保留已经发布的 ID、名称和图片;不要直接退回缺少新增 ID 的旧版本。
|
||||
|
||||
## 缓存与玩家数据
|
||||
|
||||
启动会核对清单、大小、SHA-256、JSON 契约和 ZIP 图片内容,成功后才原子切换
|
||||
数据目录下的 `catalogue/current.json`。断网、404 或版本无效时优先使用已经
|
||||
验证的缓存,再使用插件基线。日志位于同一数据目录的 `farm-preview.log`。
|
||||
|
||||
备份应包含完整数据目录及其 `catalogue` 缓存。玩家库存或地块若引用基线没有的
|
||||
新作物 ID,而可用缓存和远端均不能提供这些 ID,插件会明确拒绝启动,避免
|
||||
丢失库存。不要通过删除玩家数据解决版本问题,应恢复缓存或提供兼容发布版本。
|
||||
已播种作物继续按播种时的快照结算;出售使用当前加载版本的价格。
|
||||
|
||||
## 插件维护者刷新离线基线
|
||||
|
||||
只有准备发布新版插件时才需更新内置基线。在 Farm 根目录执行:
|
||||
|
||||
```text
|
||||
python scripts/sync_catalogue_bootstrap.py --source D:/Work/farm_service/catalogue
|
||||
python scripts/check_catalogue_bootstrap.py
|
||||
```
|
||||
|
||||
同步前请确认 Service 发布版本已审查。CI 独立核对 Farm 内置产物,不依赖相邻
|
||||
Service 工作区或在线下载。旧库导出工具只保留为显式迁移参考,不再是日常数据源。
|
||||
@@ -0,0 +1,113 @@
|
||||
# 作物目录Service与启动更新 Implementation Plan
|
||||
|
||||
> **For agentic workers:** 使用subagent-driven-development负责Service边界,
|
||||
> 主代理负责紧密耦合的Farm加载、生命周期和发布产物;最后统一只读审查。
|
||||
|
||||
**Goal:** Service独立维护完整作物资源版本,三框架仅启动检查更新并安全回退。
|
||||
**Architecture:** Service的catalogue/source为唯一可编辑源,生成不可变发布版本;
|
||||
Farm离线核心只解析目录,共享接入层下载、校验和选择版本后初始化数据库及绘图。
|
||||
**Tech Stack:** Python3.10+标准库/Pillow,Rust/Axum,SQLite,unittest/cargo test。
|
||||
**Spec:** D:/Work/farm_service/docs/crop-catalogue-design.md;用户明确要求实施。
|
||||
|
||||
## Global Constraints
|
||||
|
||||
不修改legacy、不操作真实玩家数据、不推送部署、不创建定时更新;现有已播种
|
||||
结算快照和当前价格出售规则保留。当前目录为用户此前选择的工作区;本轮不提交。
|
||||
图索引过期,已知scripts excluded/src metadata_changed,所有结论回到FastCtx源文件。
|
||||
|
||||
## Protocol / Interfaces
|
||||
|
||||
发布根catalogue/,latest位于manifest.json;历史位于releases/<release>/。
|
||||
manifest严格schema=1、version(1..64字母数字._-)、release(64位小写SHA256)、
|
||||
files;files精确四项plants.json、metadata.json、assets.zip、land-soils.zip,
|
||||
各为size_bytes正整数和sha256。release=SHA256(files对象的排序紧凑UTF8 JSON)。
|
||||
manifest<=16KiB,plants<=2MiB,metadata<=4MiB,各ZIP<=64MiB,
|
||||
解压总量<=256MiB。不得任意URL/路径,不自动跨源。
|
||||
Github/Gitea根由现有固定Service仓库raw路径拼接catalogue/。
|
||||
HTTP /v1/catalogue/manifest.json 和 /v1/catalogue/releases/{release}/{file}。
|
||||
每历史版本含相同manifest.json。Service启动读完整目录快照,运行中不重读。
|
||||
发布工具调用:python scripts/prepare_catalogue.py --source catalogue/source
|
||||
--output catalogue --version <label>;支持--check只读核对当前源与latest。
|
||||
源目录plants.json/metadata.json/images/<原ZIP成员路径>/land-soils.zip。
|
||||
plants根schema1+plants,不再依赖旧库摘要。metadata.plants从目录重建,
|
||||
images尺寸与摘要从实际图片计算,保留布局/定位/panel等现有配置。
|
||||
原子生成、不可变历史,不允许删除/更名旧ID或删除既有图片路径。
|
||||
|
||||
Farm接口:farm_core.catalogue_release提供纯离线manifest与目录一致性校验;
|
||||
farm_render.catalogue_bundle.load_catalogue_release(data_dir, source, service_url,
|
||||
required_ids=()) -> ReleaseSnapshot(catalogue, metadata, bundle_path, soils_path,
|
||||
manifest, origin),manifest/metadata可为dict,路径可在首次离线回退为空。
|
||||
完整release校验后再更新current.json指针,失败用已验证current再用bootstrap;
|
||||
所有候选必须覆盖required_ids,否则明确失败,不修改玩家库。
|
||||
缓存路径catalogue/releases/<release>/;current.json只含完整manifest;
|
||||
并发同进程锁与独立临时目录、原子替换,最终有效目录可复用不破坏。
|
||||
网络仅接入/资源层,下载限额/超时/HTTPS复用现有模式,异常只记录分类。
|
||||
bootstrap manifest位于farm_core/data/catalogue-manifest.json,其他文件沿用
|
||||
现有core/data/plants与render/data/metadata、assets、land-soils;
|
||||
三端不打包assets.zip,bootstrap无图时仍可文字启动。
|
||||
|
||||
## Review Focus
|
||||
|
||||
- 下载中断/错hash/错schema/文件混用:不切换不破坏current。
|
||||
- 旧缓存丢失但玩家持有新ID:启动拒绝,不覆盖数据库或清空库存。
|
||||
- 同时启动/取消:只采用完整版本,日志关闭后不遗留写缓存/任务。
|
||||
- 新版本禁售/禁播而非删ID:旧库存可出售、旧种植继续快照结算。
|
||||
- 公共HTTP只允许白名单文件/64位版本,保留资料鉴权与字体路由。
|
||||
|
||||
## Tasks
|
||||
|
||||
### 1. Service维护源、确定性发布与HTTP分发
|
||||
Files: service scripts/prepare_catalogue.py、catalogue/source和releases;
|
||||
src/catalogue.rs、config/lib/main与tests/catalogue.rs;docs/catalogue.md。
|
||||
- [x] 先加发布工具/HTTP失败测试,记录RED。
|
||||
- [x] 移出202条与768张现有图片及小型土地图,初始值/图不变;
|
||||
本地生成初始正式版本,不访问远端。
|
||||
- [x] Rust启动校验manifest/各文件与历史集合,公开只读快照路由,
|
||||
新环境FARM_CATALOGUE_DIR,未配置503、未知404、支持ETag。
|
||||
- [x] 运行Python工具测试、cargo fmt/test/clippy;不改Farm文件。
|
||||
|
||||
### 2. Farm纯校验、下载与bootstrap刷新
|
||||
Files: core/catalogue_release.py、render/catalogue_bundle.py、
|
||||
scripts/sync_catalogue_bootstrap.py、check_catalogue_bootstrap.py及新测试。
|
||||
- [x] 先测成功版本、非法输入、缓存回退、原子性、ID兼容和三源路径RED。
|
||||
- [x] 实现完整版本校验与缓存选择;Pillow解码只在render层。
|
||||
- [x] 显式从Service本地发布目录刷新bootstrap,CI只自检独立产物,
|
||||
不要求相邻Service或在线访问;旧导出仅作为显式迁移参考入口。
|
||||
- [x] 定向测试GREEN,不将源码检查称为HTTP发布通过。
|
||||
|
||||
### 3. 接入生命周期、数据库兼容与动态元数据
|
||||
Files: integrations/shared/runtime.py、core/storage.py、render/assets.py、
|
||||
renderer.py和既有运行时测试fixture。
|
||||
- [x] 先测启动采用新版/失败回退/旧库新ID拒绝/关闭与重复start RED。
|
||||
- [x] 启动日志后读取只读required IDs,加载版本后初始化Store/Service;
|
||||
resource_options_provider只在本次启动选择,运行中不再次检查目录。
|
||||
- [x] Assets/FarmRenderer接受metadata与soils_path,渲染绑定同版。
|
||||
- [x] 测试fixture显式mock网络边界,不能让默认单元测试访问公网。
|
||||
|
||||
### 4. 发布守卫、文档、整体审查
|
||||
Files: verify_foundation.py、export脚本CLI、package tests/README/docs。
|
||||
- [x] 替换旧库字节锁定检查为bootstrap一致性,旧基线检查独立保留。
|
||||
- [x] 三端包包含新模块/轻量bootstrap清单,拒绝大型ZIP。
|
||||
- [x] 新增新增/改价/重启/兼容回滚维护步骤;修正三端旧在线更新说明。
|
||||
- [x] 完整219+Python回归、Rust静态/集成检查及新只读审查;仅交付本地。
|
||||
|
||||
## Ledger
|
||||
|
||||
2026-10-26:开始;Farm基线e5ec38e,Service基线72e54c3。
|
||||
Comet resume-probe返回none/no-active-native-changes,按仓库约定不强建工作流。
|
||||
Ruling: 采用当前目录,保留已有未提交设计;不提交合并推送。
|
||||
Ruling: 发布四件套包含小型land-soils.zip,避免更新升级土地图后仍混用旧包。
|
||||
|
||||
2026-10-26完成:初始release为
|
||||
4c88736c2e9b1b5eec4568e953152d59a6918242802da72a6eec1a615a17f18b;
|
||||
202条作物、768张原始图片已迁入Service,初始业务值和图片字节保留。
|
||||
Farm完整verify_foundation退出0:238项,237通过、1项NoneBot/Pydantic
|
||||
宿主配置依赖跳过;工作流守卫和两仓库git diff --check通过。
|
||||
Service发布器18项、Rust集成16项通过,fmt/clippy通过,初始--check通过。
|
||||
两侧独立定向审查完成;修复了完整展示契约、并发发布、名称契约、超大整数
|
||||
溢出和启动关闭交错问题,新增回归先复现失败后通过。文档自检命令已补version。
|
||||
Farm测试日志:runtime/catalogue-verification.log。
|
||||
验证边界:Windows/Python3.12.14/Pillow12.3;未执行Linux系统锁分支、
|
||||
其他依赖矩阵、真实宿主、生产数据库、远端下载、提交、推送或部署。
|
||||
字体及旧素材下载线程的取消等待属于既有实现边界,本次新作物版本加载线程
|
||||
已独立跟踪并在关闭时等待排空,不宣称旧资源线程也具备同等保证。
|
||||
@@ -12,7 +12,8 @@
|
||||
当前支持注册、农场/地块/详情、种子商店、仓库、购买、播种、收获、
|
||||
铲除、加锁/解锁、出售,以及已接入的土地操作、虫害和偷菜命令。
|
||||
实际参数以“农场帮助”为准。新账号初始500农场币、6块普通空地。
|
||||
签到、活动和作物目录在线更新尚未实现;图片/字体下载不等于作物更新。
|
||||
签到和活动尚未实现。作物目录已迁入 Service,插件启动时检查更新;
|
||||
维护与回滚见 [作物数据维护](catalogue-updates.md)。
|
||||
|
||||
## 构建和验证
|
||||
|
||||
|
||||
@@ -23,6 +23,7 @@ from ._vendor.farm_render import FarmRenderer, RenderOptions, RenderService
|
||||
from ._vendor.farm_render.seasons import season_for
|
||||
from ._vendor.farm_render.bundle import ensure_assets, validate_options
|
||||
from ._vendor.farm_render.font_bundle import ensure_font
|
||||
from ._vendor.farm_render.catalogue_bundle import load_catalogue_release
|
||||
from ._vendor.farm_render.shop import shop_page
|
||||
from ._vendor.farm_render.details import detail_rows, details_text
|
||||
from ._vendor.farm_render.warehouse import (
|
||||
@@ -136,12 +137,21 @@ class Runtime:
|
||||
self._resource_lock = asyncio.Lock()
|
||||
self._closing = False
|
||||
self._close_task = None
|
||||
self._start_lock = asyncio.Lock()
|
||||
self._catalogue_task = None
|
||||
self.catalogue_release = None
|
||||
self._start_epoch = 0
|
||||
|
||||
async def start(self):
|
||||
async with self._start_lock:
|
||||
await self._start()
|
||||
|
||||
async def _start(self):
|
||||
if self._closing:
|
||||
return
|
||||
if self.started:
|
||||
return
|
||||
epoch = self._start_epoch
|
||||
self.data_dir.mkdir(parents=True, exist_ok=True)
|
||||
self.log_handler = logging.FileHandler(
|
||||
self.data_dir / "farm-preview.log", encoding="utf-8",
|
||||
@@ -152,12 +162,34 @@ class Runtime:
|
||||
logging.getLogger(__package__).setLevel(logging.INFO)
|
||||
logging.getLogger(__package__).addHandler(self.log_handler)
|
||||
try:
|
||||
if self.resource_options_provider is not None:
|
||||
self.resource_source, self.service_url = (
|
||||
self.resource_options_provider()
|
||||
)
|
||||
validate_options(self.resource_source, self.service_url)
|
||||
required_ids = await self.store.catalogue_ids()
|
||||
if self._closing or epoch != self._start_epoch:
|
||||
return
|
||||
self._catalogue_task = asyncio.create_task(asyncio.to_thread(
|
||||
load_catalogue_release, self.data_dir, self.resource_source,
|
||||
self.service_url, required_ids,
|
||||
))
|
||||
self.catalogue_release = await asyncio.shield(self._catalogue_task)
|
||||
if self._closing or epoch != self._start_epoch:
|
||||
return
|
||||
self.service = FarmService(
|
||||
self.store, self.catalogue_release.catalogue,
|
||||
)
|
||||
await self.store.initialize(catalogue=self.service.catalogue)
|
||||
except Exception:
|
||||
if self._closing or epoch != self._start_epoch:
|
||||
return
|
||||
except BaseException:
|
||||
logger.error("farm preview database initialization failed")
|
||||
await self.close()
|
||||
raise
|
||||
await self._prepare_resources()
|
||||
if self._closing or epoch != self._start_epoch:
|
||||
return
|
||||
self.started = True
|
||||
logger.info("farm preview initialized schema=%s", SCHEMA_VERSION)
|
||||
|
||||
@@ -167,16 +199,20 @@ class Runtime:
|
||||
return
|
||||
resource = 'configuration'
|
||||
try:
|
||||
if self.resource_options_provider is not None:
|
||||
if (self.catalogue_release is None
|
||||
and self.resource_options_provider is not None):
|
||||
self.resource_source, self.service_url = (
|
||||
self.resource_options_provider()
|
||||
)
|
||||
validate_options(self.resource_source, self.service_url)
|
||||
resource = 'assets'
|
||||
bundle_path = await asyncio.to_thread(
|
||||
ensure_assets, self.data_dir, self.resource_source,
|
||||
self.service_url,
|
||||
)
|
||||
selected = self.catalogue_release
|
||||
bundle_path = selected.bundle_path if selected else None
|
||||
if bundle_path is None:
|
||||
bundle_path = await asyncio.to_thread(
|
||||
ensure_assets, self.data_dir, self.resource_source,
|
||||
self.service_url,
|
||||
)
|
||||
resource = 'font'
|
||||
candidate = (Path(self.font_path) if self.font_path else
|
||||
await asyncio.to_thread(
|
||||
@@ -186,6 +222,8 @@ class Runtime:
|
||||
resource = 'renderer'
|
||||
self.rendering = RenderService(FarmRenderer(
|
||||
candidate, bundle_path=bundle_path,
|
||||
metadata=selected.metadata if selected else None,
|
||||
soils_path=selected.soils_path if selected else None,
|
||||
))
|
||||
logger.info('farm resources ready font=%s', candidate.name)
|
||||
except Exception as error:
|
||||
@@ -202,6 +240,7 @@ class Runtime:
|
||||
async def close(self):
|
||||
if self._close_task is None or self._close_task.done():
|
||||
self._closing = True
|
||||
self._start_epoch += 1
|
||||
self._close_task = asyncio.create_task(self._finish_close())
|
||||
task = self._close_task
|
||||
try:
|
||||
@@ -213,6 +252,11 @@ class Runtime:
|
||||
|
||||
async def _finish_close(self):
|
||||
try:
|
||||
if self._catalogue_task is not None:
|
||||
try:
|
||||
await asyncio.shield(self._catalogue_task)
|
||||
except Exception:
|
||||
pass
|
||||
async with self._resource_lock:
|
||||
if self.rendering:
|
||||
await self.rendering.close()
|
||||
|
||||
+1
-1
@@ -65,7 +65,7 @@
|
||||
- [x] 完善我的农场图片,例如左上角显示用户数据
|
||||
- [ ] 完善升级数据、作物数据、作物图片
|
||||
- [x] 签到功能
|
||||
- [x] 在线更新作物信息
|
||||
- [x] 作物信息启动时检查更新(Service维护,失败回退已验证缓存)
|
||||
- [ ] 添加渔场功能
|
||||
- [ ] 增加活动、交易行功能
|
||||
- [ ] 增加交易行总行功能
|
||||
|
||||
+1
-1
@@ -65,7 +65,7 @@
|
||||
- [x] 完善我的农场图片,例如左上角显示用户数据
|
||||
- [ ] 完善升级数据、作物数据、作物图片
|
||||
- [x] 签到功能
|
||||
- [x] 在线更新作物信息
|
||||
- [x] 作物信息启动时检查更新(Service维护,失败回退已验证缓存)
|
||||
- [ ] 添加渔场功能
|
||||
- [ ] 增加活动、交易行功能
|
||||
- [ ] 增加交易行总行功能
|
||||
|
||||
+1
-1
@@ -81,7 +81,7 @@ farm-preview.log;旧farm.db不得直接改名覆盖。资源源默认Github,
|
||||
- [x] 完善我的农场图片,例如左上角显示用户数据
|
||||
- [ ] 完善升级数据、作物数据、作物图片
|
||||
- [ ] 签到功能
|
||||
- [ ] 在线更新作物信息(当前作物目录仍随插件内置)
|
||||
- [x] 作物信息启动时检查更新(Service维护,失败回退已验证缓存)
|
||||
- [ ] 添加渔场功能
|
||||
- [ ] 增加活动、交易行功能
|
||||
- [ ] 增加交易行总行功能
|
||||
|
||||
@@ -3,8 +3,14 @@
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
from zipfile import ZipFile
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / 'src'))
|
||||
from farm_core.catalogue_release import parse_manifest
|
||||
from farm_render.catalogue_bundle import ROOTS
|
||||
|
||||
|
||||
def check(root):
|
||||
data = Path(root) / 'src/farm_render/data'
|
||||
@@ -19,12 +25,13 @@ def check(root):
|
||||
with ZipFile(bundle) as archive:
|
||||
if set(archive.namelist()) != set(metadata['images']):
|
||||
raise ValueError('Asset manifest and layout differ')
|
||||
for source, prefix in (
|
||||
('github', 'https://raw.githubusercontent.com/Shu-Ying/'
|
||||
'farm_service/main/assets/assets.zip'),
|
||||
('gitea', 'https://gitea.diuse.work/Diuse/farm_service/'
|
||||
'raw/branch/main/assets/assets.zip'),
|
||||
):
|
||||
bootstrap = parse_manifest((Path(root) / 'src/farm_core/data/'
|
||||
'catalogue-manifest.json').read_bytes())
|
||||
if manifest.get('catalogue_release') != bootstrap['release']:
|
||||
raise ValueError('Bootstrap and asset release differ')
|
||||
for name, base in ROOTS.items():
|
||||
source = name.lower()
|
||||
prefix = base + '/releases/' + bootstrap['release'] + '/assets.zip'
|
||||
if manifest['sources'][source] != prefix:
|
||||
raise ValueError('Asset source must use service repository')
|
||||
print('Verified external asset manifest and development ZIP')
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
"""CI只核对插件初始版本自身,不访问远端或相邻Service开发仓库。"""
|
||||
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / 'src'))
|
||||
|
||||
from farm_core.catalogue_release import parse_manifest
|
||||
from farm_render.catalogue_bundle import _validate
|
||||
|
||||
|
||||
def check(root=ROOT):
|
||||
root = Path(root)
|
||||
core = root / 'src/farm_core/data'
|
||||
render = root / 'src/farm_render/data'
|
||||
manifest = parse_manifest((core / 'catalogue-manifest.json').read_bytes())
|
||||
content = {'plants.json': (core / 'plants.json').read_bytes(),
|
||||
**{name: (render / name).read_bytes() for name in (
|
||||
'metadata.json', 'assets.zip', 'land-soils.zip',
|
||||
)}}
|
||||
_validate(content, manifest, None, ())
|
||||
print('Verified independent catalogue bootstrap ' + manifest['release'])
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
check()
|
||||
@@ -37,14 +37,24 @@ def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument(
|
||||
"--source", type=Path,
|
||||
default=root / "legacy/zhenxun_plugin_farm/resource/db/plant.db",
|
||||
default=None,
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output", type=Path,
|
||||
default=root / "src/farm_core/data/plants.json",
|
||||
default=None,
|
||||
)
|
||||
parser.add_argument("--check", action="store_true")
|
||||
parser.add_argument('--legacy-reference', action='store_true')
|
||||
args = parser.parse_args()
|
||||
if args.check and args.source is None:
|
||||
from check_catalogue_bootstrap import check
|
||||
check(root)
|
||||
return
|
||||
if not args.legacy_reference or args.output is None:
|
||||
parser.error('Daily updates use sync_catalogue_bootstrap.py; '
|
||||
'legacy export requires --legacy-reference --output')
|
||||
if args.source is None:
|
||||
args.source = root / 'legacy/zhenxun_plugin_farm/resource/db/plant.db'
|
||||
content = export(args.source)
|
||||
if args.check:
|
||||
if not args.output.is_file() or args.output.read_bytes() != content:
|
||||
|
||||
@@ -119,9 +119,10 @@ def export(root):
|
||||
) for theme in ("default", "spring", "summer", "autumn", "winter", "bee")]
|
||||
paths += list((source / "background/decorateIcon").glob("decorate_*.png"))
|
||||
layout = json.loads((legacy / "config/soil.json").read_bytes())
|
||||
catalogue = json.loads((
|
||||
root / "src/farm_core/data/plants.json"
|
||||
).read_bytes())
|
||||
from export_catalogue import export as export_legacy_catalogue
|
||||
catalogue = json.loads(export_legacy_catalogue(
|
||||
legacy / 'resource/db/plant.db',
|
||||
))
|
||||
metadata = {
|
||||
"schema": 1, "canvas": [3072, 1872], "soil_size": layout["size"],
|
||||
"positions": [layout["soil"][str(i)] for i in range(1, 31)],
|
||||
@@ -188,10 +189,19 @@ def main():
|
||||
root = Path(__file__).resolve().parents[1]
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--check", action="store_true")
|
||||
parser.add_argument('--legacy-reference', action='store_true')
|
||||
parser.add_argument('--output', type=Path)
|
||||
args = parser.parse_args()
|
||||
if args.check and not args.legacy_reference:
|
||||
from check_catalogue_bootstrap import check
|
||||
check(root)
|
||||
return
|
||||
if not args.legacy_reference or args.output is None:
|
||||
parser.error('Daily updates use Service prepare_catalogue.py; '
|
||||
'legacy export requires --legacy-reference --output')
|
||||
files = (*zip(("assets.zip", "metadata.json"), export(root)),
|
||||
('land-soils.zip', export_soils(root)))
|
||||
destination = root / "src/farm_render/data"
|
||||
destination = args.output
|
||||
for name, content in files:
|
||||
path = destination / name
|
||||
if args.check:
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
"""显式从已验证Service发布目录刷新插件初始版本,不联网。"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import sys
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / 'src'))
|
||||
|
||||
from farm_core import Catalogue
|
||||
from farm_core.catalogue_release import canonical, parse_manifest
|
||||
from farm_render.catalogue_bundle import ROOTS, _snapshot
|
||||
|
||||
|
||||
def sync(source, root=ROOT):
|
||||
source, root = Path(source), Path(root)
|
||||
manifest = parse_manifest((source / 'manifest.json').read_bytes())
|
||||
release = source / 'releases' / manifest['release']
|
||||
snapshot = _snapshot(release, manifest, Catalogue.bundled())
|
||||
destinations = {
|
||||
'plants.json': root / 'src/farm_core/data/plants.json',
|
||||
**{name: root / 'src/farm_render/data' / name for name in (
|
||||
'metadata.json', 'assets.zip', 'land-soils.zip',
|
||||
)},
|
||||
}
|
||||
for name, path in destinations.items():
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copyfile(release / name, path)
|
||||
data = root / 'src/farm_core/data'
|
||||
(data / 'catalogue-manifest.json').write_bytes(canonical(manifest) + b'\n')
|
||||
asset = {
|
||||
'schema': 1, 'version': manifest['version'],
|
||||
'sha256': manifest['files']['assets.zip']['sha256'],
|
||||
'size_bytes': manifest['files']['assets.zip']['size_bytes'],
|
||||
'catalogue_release': manifest['release'],
|
||||
'sources': {
|
||||
name.lower(): base + '/releases/' + manifest['release']
|
||||
+ '/assets.zip' for name, base in ROOTS.items()
|
||||
},
|
||||
}
|
||||
asset['sources']['farm_service'] = (
|
||||
'https://farm.example/v1/catalogue/releases/'
|
||||
+ manifest['release'] + '/assets.zip'
|
||||
)
|
||||
(root / 'src/farm_render/data/asset-manifest.json').write_text(
|
||||
json.dumps(asset, ensure_ascii=False, indent=2) + '\n',
|
||||
encoding='utf-8',
|
||||
)
|
||||
return snapshot.manifest
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--source', type=Path, required=True,
|
||||
help='Service catalogue发布目录,不是可编辑source目录')
|
||||
args = parser.parse_args()
|
||||
print('Bootstrap synced release=' + sync(args.source)['release'])
|
||||
@@ -14,8 +14,7 @@ def main():
|
||||
}
|
||||
commands = (
|
||||
[sys.executable, "scripts/check_baseline.py"],
|
||||
[sys.executable, "scripts/export_catalogue.py", "--check"],
|
||||
[sys.executable, "scripts/export_render_assets.py", "--check"],
|
||||
[sys.executable, 'scripts/check_catalogue_bootstrap.py'],
|
||||
[sys.executable, 'scripts/check_asset_manifest.py'],
|
||||
[
|
||||
sys.executable, "-m", "unittest", "discover",
|
||||
|
||||
@@ -0,0 +1,110 @@
|
||||
"""作物发布协议的纯离线校验;不联网、不依赖绘图库。"""
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import re
|
||||
|
||||
from .catalogue import Catalogue
|
||||
|
||||
FILES = {
|
||||
'plants.json': 2 * 1024 * 1024,
|
||||
'metadata.json': 4 * 1024 * 1024,
|
||||
'assets.zip': 64 * 1024 * 1024,
|
||||
'land-soils.zip': 64 * 1024 * 1024,
|
||||
}
|
||||
MANIFEST_LIMIT = 16 * 1024
|
||||
DIGEST = re.compile(r'[0-9a-f]{64}')
|
||||
VERSION = re.compile(r'[A-Za-z0-9._-]{1,64}')
|
||||
|
||||
|
||||
def _pairs(items):
|
||||
result = {}
|
||||
for key, value in items:
|
||||
if key in result:
|
||||
raise ValueError('Duplicate JSON key')
|
||||
result[key] = value
|
||||
return result
|
||||
|
||||
|
||||
def decode(content):
|
||||
def invalid(_value):
|
||||
raise ValueError('Non-finite JSON value')
|
||||
try:
|
||||
return json.loads(content, object_pairs_hook=_pairs,
|
||||
parse_constant=invalid)
|
||||
except (UnicodeError, TypeError, RecursionError) as error:
|
||||
raise ValueError('Invalid release JSON') from error
|
||||
|
||||
|
||||
def canonical(value):
|
||||
return json.dumps(value, sort_keys=True, ensure_ascii=False,
|
||||
separators=(',', ':'), allow_nan=False).encode()
|
||||
|
||||
|
||||
def digest(content):
|
||||
return hashlib.sha256(content).hexdigest()
|
||||
|
||||
|
||||
def make_manifest(files, version):
|
||||
descriptors = {
|
||||
name: {'size_bytes': len(content), 'sha256': digest(content)}
|
||||
for name, content in files.items()
|
||||
}
|
||||
result = {'schema': 1, 'version': version,
|
||||
'release': digest(canonical(descriptors)), 'files': descriptors}
|
||||
return parse_manifest(canonical(result))
|
||||
|
||||
|
||||
def parse_manifest(content):
|
||||
if len(content) > MANIFEST_LIMIT:
|
||||
raise ValueError('Catalogue manifest exceeds limit')
|
||||
value = decode(content)
|
||||
if (type(value) is not dict or set(value) != {
|
||||
'schema', 'version', 'release', 'files'}
|
||||
or type(value['schema']) is not int or value['schema'] != 1
|
||||
or not isinstance(value['version'], str)
|
||||
or VERSION.fullmatch(value['version']) is None
|
||||
or not isinstance(value['release'], str)
|
||||
or DIGEST.fullmatch(value['release']) is None
|
||||
or type(value['files']) is not dict
|
||||
or set(value['files']) != set(FILES)):
|
||||
raise ValueError('Invalid catalogue manifest')
|
||||
for name, limit in FILES.items():
|
||||
item = value['files'][name]
|
||||
if (type(item) is not dict or set(item) != {'size_bytes', 'sha256'}
|
||||
or type(item['size_bytes']) is not int
|
||||
or not 0 < item['size_bytes'] <= limit
|
||||
or not isinstance(item['sha256'], str)
|
||||
or DIGEST.fullmatch(item['sha256']) is None):
|
||||
raise ValueError('Invalid catalogue file descriptor')
|
||||
if digest(canonical(value['files'])) != value['release']:
|
||||
raise ValueError('Catalogue release fingerprint differs')
|
||||
return value
|
||||
|
||||
|
||||
def check_file(content, descriptor):
|
||||
if (len(content) != descriptor['size_bytes']
|
||||
or digest(content) != descriptor['sha256']):
|
||||
raise ValueError('Catalogue file size or SHA256 differs')
|
||||
|
||||
|
||||
def load_catalogue(content, previous=None, required_ids=()):
|
||||
value = decode(content)
|
||||
if (type(value) is not dict or type(value.get('plants')) is not list
|
||||
or not 0 < len(value['plants']) <= 10000):
|
||||
raise ValueError('Invalid catalogue document')
|
||||
catalogue = Catalogue.loads(content)
|
||||
names = [plant.name for plant in catalogue]
|
||||
if (len(set(names)) != len(names) or any(
|
||||
name in ('.', '..') or '/' in name or '\\' in name
|
||||
for name in names)):
|
||||
raise ValueError('Invalid or duplicate catalogue name')
|
||||
indexed = {plant.plant_id: plant for plant in catalogue}
|
||||
if not set(required_ids).issubset(indexed):
|
||||
raise ValueError('catalogue missing player crop ID')
|
||||
if previous is not None:
|
||||
for plant in previous:
|
||||
current = indexed.get(plant.plant_id)
|
||||
if current is None or current.name != plant.name:
|
||||
raise ValueError('Catalogue removed or renamed crop ID')
|
||||
return catalogue
|
||||
@@ -0,0 +1 @@
|
||||
{"files":{"assets.zip":{"sha256":"78d6e8e448fd5296b21d471b98ea7d58c81ef78a82b0e192a1ed041bb5fddad9","size_bytes":21264561},"land-soils.zip":{"sha256":"e27151f0b92c36842c23d9bf3d4855e786ef5a9db37c736e751b53be8d3ebda9","size_bytes":57889},"metadata.json":{"sha256":"6ba1c2134d1a853cd4c2bfd6489e68bd59a0c6068cd79289a1f4bcaa62f2444d","size_bytes":197449},"plants.json":{"sha256":"e05e4430643710ccbeeeb1c7c8640ec5962ee097de6b3d5fad0fb7860c596a2b","size_bytes":87259}},"release":"4c88736c2e9b1b5eec4568e953152d59a6918242802da72a6eec1a615a17f18b","schema":1,"version":"catalogue-20261026-1"}
|
||||
@@ -4445,6 +4445,5 @@
|
||||
"vipBuy": 0
|
||||
}
|
||||
],
|
||||
"schema": 1,
|
||||
"source_sha256": "21f11e818909dda4d60864374062c46e0c86e9c2f01d2759225aba466d9de9eb"
|
||||
"schema": 1
|
||||
}
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
"""跨进程发布/缓存锁;进程退出由操作系统释放,不使用永久锁标记。"""
|
||||
|
||||
from contextlib import contextmanager
|
||||
import os
|
||||
from pathlib import Path
|
||||
import time
|
||||
|
||||
|
||||
@contextmanager
|
||||
def release_lock(path, timeout=180):
|
||||
path = Path(path)
|
||||
with path.open('a+b') as stream:
|
||||
if stream.seek(0, os.SEEK_END) == 0:
|
||||
stream.write(b'0')
|
||||
stream.flush()
|
||||
deadline = time.monotonic() + timeout
|
||||
if os.name == 'nt':
|
||||
import msvcrt
|
||||
def acquire():
|
||||
stream.seek(0)
|
||||
msvcrt.locking(stream.fileno(), msvcrt.LK_NBLCK, 1)
|
||||
def unlock():
|
||||
stream.seek(0)
|
||||
msvcrt.locking(stream.fileno(), msvcrt.LK_UNLCK, 1)
|
||||
else:
|
||||
import fcntl
|
||||
def acquire():
|
||||
fcntl.flock(stream.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
def unlock():
|
||||
fcntl.flock(stream.fileno(), fcntl.LOCK_UN)
|
||||
while True:
|
||||
try:
|
||||
acquire()
|
||||
break
|
||||
except OSError:
|
||||
if time.monotonic() >= deadline:
|
||||
raise TimeoutError('Catalogue cache is busy') from None
|
||||
time.sleep(0.05)
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
unlock()
|
||||
@@ -103,6 +103,50 @@ SCHEMA = (
|
||||
|
||||
|
||||
class SQLiteStore:
|
||||
async def catalogue_ids(self):
|
||||
"""初始化前只读检查库存与地块ID;不创建文件或迁移数据库。"""
|
||||
task = asyncio.create_task(asyncio.to_thread(self._catalogue_ids))
|
||||
try:
|
||||
return await asyncio.shield(task)
|
||||
except asyncio.CancelledError:
|
||||
try:
|
||||
await task
|
||||
except Exception:
|
||||
pass
|
||||
raise
|
||||
|
||||
def _catalogue_ids(self):
|
||||
if not self.path.is_file():
|
||||
return set()
|
||||
with closing(sqlite3.connect(
|
||||
self.path.as_uri() + '?mode=ro', uri=True)) as connection:
|
||||
connection.execute('BEGIN')
|
||||
app = connection.execute('PRAGMA application_id').fetchone()[0]
|
||||
version = connection.execute('PRAGMA user_version').fetchone()[0]
|
||||
tables = {row[0] for row in connection.execute(
|
||||
"SELECT name FROM sqlite_master WHERE type='table'",
|
||||
)}
|
||||
if app == 0 and version == 0 and not tables:
|
||||
return set()
|
||||
if app != APPLICATION_ID or not 1 <= version <= SCHEMA_VERSION:
|
||||
raise SchemaError('Unsupported farm database')
|
||||
result = set()
|
||||
if version >= 2:
|
||||
result.update(row[0] for row in connection.execute(
|
||||
'SELECT DISTINCT plant_id FROM inventory '
|
||||
'WHERE seeds>0 OR crops>0',
|
||||
))
|
||||
for row in connection.execute(
|
||||
'SELECT plant FROM plots WHERE plant IS NOT NULL'):
|
||||
try:
|
||||
result.add(json.loads(row[0])['plant_id'])
|
||||
except (KeyError, TypeError, ValueError) as error:
|
||||
raise SchemaError('Invalid saved crop definition') \
|
||||
from error
|
||||
if any(type(value) is not int or value <= 0 for value in result):
|
||||
raise SchemaError('Invalid saved crop ID')
|
||||
return result
|
||||
|
||||
def __init__(self, path: str | Path, timeout: float = 10):
|
||||
if str(path) == ":memory:":
|
||||
raise ValueError("Use a dedicated file for transaction connections")
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
"""包内素材与有界 LRU;不下载、不解压到磁盘、不暴露共享图片。"""
|
||||
|
||||
from collections import OrderedDict
|
||||
from copy import deepcopy
|
||||
from importlib import resources
|
||||
from io import BytesIO
|
||||
import json
|
||||
@@ -42,10 +43,14 @@ class ByteLRU:
|
||||
|
||||
|
||||
class Assets:
|
||||
def __init__(self, cache_bytes=32 * 1024 * 1024, bundle_path=None):
|
||||
def __init__(self, cache_bytes=32 * 1024 * 1024, bundle_path=None,
|
||||
metadata=None, soils_path=None):
|
||||
data = resources.files(__package__).joinpath("data")
|
||||
self.metadata = json.loads(data.joinpath("metadata.json").read_bytes())
|
||||
self.metadata = (deepcopy(metadata) if metadata is not None else
|
||||
json.loads(data.joinpath('metadata.json')
|
||||
.read_bytes()))
|
||||
self.cache = ByteLRU(cache_bytes)
|
||||
self._soils = soils_path or data.joinpath('land-soils.zip')
|
||||
self._bundle = (bundle_path if bundle_path is not None
|
||||
else data.joinpath('assets.zip'))
|
||||
|
||||
@@ -58,9 +63,7 @@ class Assets:
|
||||
key = (name, size)
|
||||
cached = self.cache.get(key)
|
||||
if cached is None:
|
||||
source = (resources.files(__package__).joinpath(
|
||||
'data', 'land-soils.zip',
|
||||
) if upgrade else self._bundle)
|
||||
source = self._soils if upgrade else self._bundle
|
||||
with source.open("rb") as stream, ZipFile(stream) as bundle:
|
||||
content = bundle.read(name)
|
||||
with Image.open(BytesIO(content)) as original:
|
||||
|
||||
@@ -99,8 +99,12 @@ def ensure_assets(data_dir, source='Github', service_url=''):
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
# 清单绑定在插件内;远端不能通过更换清单放行不兼容的素材。
|
||||
if source == 'farm_service':
|
||||
url = (service_url.rstrip('/') + '/v1/assets/'
|
||||
+ expected['sha256'] + '/assets.zip')
|
||||
if 'catalogue_release' in expected:
|
||||
url = (service_url.rstrip('/') + '/v1/catalogue/releases/'
|
||||
+ expected['catalogue_release'] + '/assets.zip')
|
||||
else:
|
||||
url = (service_url.rstrip('/') + '/v1/assets/'
|
||||
+ expected['sha256'] + '/assets.zip')
|
||||
else:
|
||||
url = expected['sources'][source.lower()]
|
||||
temporary = None
|
||||
|
||||
@@ -0,0 +1,208 @@
|
||||
"""启动时选择完整作物版本;网络失败保留已验证缓存或初始目录。"""
|
||||
|
||||
from dataclasses import dataclass, replace
|
||||
from importlib import resources
|
||||
from io import BytesIO
|
||||
import logging
|
||||
from pathlib import Path
|
||||
import tempfile
|
||||
import threading
|
||||
import uuid
|
||||
from zipfile import BadZipFile
|
||||
|
||||
from farm_core.catalogue_release import (
|
||||
FILES, MANIFEST_LIMIT, canonical, check_file, decode, load_catalogue,
|
||||
parse_manifest,
|
||||
)
|
||||
from farm_core.domain import FarmError
|
||||
from farm_core.release_lock import release_lock
|
||||
|
||||
from .bundle import _copy_remote, validate_options
|
||||
from .release_validation import validate_archive, validate_metadata
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
_LOCK = threading.RLock()
|
||||
_FAILURES = (OSError, ValueError, FarmError, EOFError, BadZipFile)
|
||||
ROOTS = {
|
||||
'Github': 'https://raw.githubusercontent.com/Shu-Ying/'
|
||||
'farm_service/main/catalogue',
|
||||
'Gitea': 'https://gitea.diuse.work/Diuse/farm_service/'
|
||||
'raw/branch/main/catalogue',
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ReleaseSnapshot:
|
||||
catalogue: object # 完整且已校验的离线业务目录。
|
||||
metadata: dict # 与目录和图片绑定的展示配置。
|
||||
bundle_path: Path | None # 无图片缓存的离线初始启动可为空。
|
||||
soils_path: Path # 小型升级土地资源所在路径。
|
||||
manifest: dict # 本次采用的发布版本和四文件摘要。
|
||||
origin: str # remote、cache或bootstrap,仅用于安全日志。
|
||||
|
||||
|
||||
def _fetch(url, limit):
|
||||
output = BytesIO()
|
||||
_copy_remote(url, output, limit)
|
||||
return output.getvalue()
|
||||
|
||||
|
||||
def _read(path, limit):
|
||||
with path.open('rb') as stream:
|
||||
content = stream.read(limit + 1)
|
||||
if len(content) > limit:
|
||||
raise ValueError('Catalogue file exceeds limit')
|
||||
return content
|
||||
|
||||
|
||||
def _atomic(path, content):
|
||||
temporary = None
|
||||
try:
|
||||
with tempfile.NamedTemporaryFile(
|
||||
dir=path.parent, suffix='.download', delete=False) as stream:
|
||||
temporary = Path(stream.name)
|
||||
stream.write(content)
|
||||
temporary.replace(path)
|
||||
finally:
|
||||
if temporary is not None:
|
||||
temporary.unlink(missing_ok=True)
|
||||
|
||||
|
||||
def _validate(content, manifest, previous, required_ids):
|
||||
for name, value in content.items():
|
||||
check_file(value, manifest['files'][name])
|
||||
catalogue = load_catalogue(content['plants.json'], previous, required_ids)
|
||||
metadata = validate_metadata(content['metadata.json'], catalogue)
|
||||
for row in decode(content['plants.json'])['plants']:
|
||||
info = metadata['plants'][row['name']]
|
||||
if (info['phase'] != row['phase']
|
||||
or info.get('general') != row.get('general')):
|
||||
raise ValueError('Catalogue and render growth definitions differ')
|
||||
if 'assets.zip' in content:
|
||||
validate_archive(content['assets.zip'], metadata['images'])
|
||||
validate_archive(content['land-soils.zip'])
|
||||
return catalogue, metadata
|
||||
|
||||
|
||||
def _snapshot(root, manifest, previous=None, required_ids=(), origin='cache'):
|
||||
content = {name: _read(root / name, checks['size_bytes'])
|
||||
for name, checks in manifest['files'].items()}
|
||||
catalogue, metadata = _validate(
|
||||
content, manifest, previous, required_ids,
|
||||
)
|
||||
return ReleaseSnapshot(catalogue, metadata, root / 'assets.zip',
|
||||
root / 'land-soils.zip', manifest, origin)
|
||||
|
||||
|
||||
def _bootstrap(required_ids=()):
|
||||
core_name = ('farm_core' if __package__ == 'farm_render' else
|
||||
__package__.rsplit('.', 1)[0] + '.farm_core')
|
||||
core = resources.files(core_name).joinpath('data')
|
||||
render = resources.files(__package__).joinpath('data')
|
||||
manifest = parse_manifest(core.joinpath(
|
||||
'catalogue-manifest.json',
|
||||
).read_bytes())
|
||||
content = {
|
||||
'plants.json': core.joinpath('plants.json').read_bytes(),
|
||||
'metadata.json': render.joinpath('metadata.json').read_bytes(),
|
||||
'land-soils.zip': render.joinpath('land-soils.zip').read_bytes(),
|
||||
}
|
||||
catalogue, metadata = _validate(content, manifest, None, required_ids)
|
||||
return ReleaseSnapshot(catalogue, metadata, None,
|
||||
Path(str(render.joinpath('land-soils.zip'))),
|
||||
manifest, 'bootstrap')
|
||||
|
||||
|
||||
def load_catalogue_release(data_dir, source='Github', service_url='',
|
||||
required_ids=()):
|
||||
"""每次调用只检查一次最新版本;调用方仅在启动时调用。"""
|
||||
validate_options(source, service_url)
|
||||
base = (service_url.rstrip('/') + '/v1/catalogue'
|
||||
if source == 'farm_service' else ROOTS[source])
|
||||
root = Path(data_dir) / 'catalogue'
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
with _LOCK, release_lock(root / '.cache.lock'):
|
||||
releases = root / 'releases'
|
||||
releases.mkdir(exist_ok=True)
|
||||
bootstrap = _bootstrap()
|
||||
current = None
|
||||
try:
|
||||
manifest = parse_manifest(_read(root / 'current.json',
|
||||
MANIFEST_LIMIT))
|
||||
current = _snapshot(
|
||||
releases / manifest['release'], manifest,
|
||||
bootstrap.catalogue,
|
||||
)
|
||||
except _FAILURES:
|
||||
pass
|
||||
previous = (current or bootstrap).catalogue
|
||||
try:
|
||||
manifest = parse_manifest(_fetch(
|
||||
base + '/manifest.json', MANIFEST_LIMIT,
|
||||
))
|
||||
target = releases / manifest['release']
|
||||
if (current is not None
|
||||
and current.manifest['release'] == manifest['release']):
|
||||
if not set(required_ids).issubset(
|
||||
plant.plant_id for plant in current.catalogue):
|
||||
raise ValueError('catalogue missing player crop ID')
|
||||
return replace(current, origin='remote')
|
||||
try:
|
||||
candidate = _snapshot(target, manifest, previous, required_ids,
|
||||
'remote')
|
||||
except _FAILURES:
|
||||
with tempfile.TemporaryDirectory(
|
||||
dir=releases, prefix='.download-') as temporary:
|
||||
staging = Path(temporary)
|
||||
for name, item in manifest['files'].items():
|
||||
content = _fetch(
|
||||
base + '/releases/' + manifest['release']
|
||||
+ '/' + name, item['size_bytes'],
|
||||
)
|
||||
check_file(content, item)
|
||||
(staging / name).write_bytes(content)
|
||||
_snapshot(staging, manifest, previous, required_ids)
|
||||
if target.exists():
|
||||
try:
|
||||
_snapshot(target, manifest, previous, required_ids)
|
||||
except _FAILURES:
|
||||
target.rename(releases / (
|
||||
'.invalid-' + uuid.uuid4().hex
|
||||
))
|
||||
if not target.exists():
|
||||
try:
|
||||
staging.rename(target)
|
||||
except OSError:
|
||||
if not target.exists():
|
||||
raise
|
||||
candidate = _snapshot(target, manifest, previous,
|
||||
required_ids, 'remote')
|
||||
previous_images = (current or bootstrap).metadata['images']
|
||||
if not set(previous_images).issubset(candidate.metadata['images']):
|
||||
raise ValueError('Catalogue removed existing image paths')
|
||||
_atomic(root / 'current.json', canonical(manifest))
|
||||
logger.info('farm catalogue ready release=%s source=%s',
|
||||
manifest['release'], source)
|
||||
return candidate
|
||||
except _FAILURES as error:
|
||||
logger.warning('farm catalogue update unavailable source=%s '
|
||||
'reason=%s; keeping verified fallback',
|
||||
source, type(error).__name__)
|
||||
if current is not None:
|
||||
if not set(required_ids).issubset(
|
||||
plant.plant_id for plant in current.catalogue):
|
||||
raise ValueError('catalogue missing player crop ID')
|
||||
return current
|
||||
bootstrap = _bootstrap(required_ids)
|
||||
# 若旧方式图片缓存可用,离线初始版本也能继续绘图。
|
||||
cached = (Path(data_dir) / 'assets'
|
||||
/ bootstrap.manifest['files']['assets.zip']['sha256']
|
||||
/ 'assets.zip')
|
||||
try:
|
||||
content = _read(cached, FILES['assets.zip'])
|
||||
check_file(content, bootstrap.manifest['files']['assets.zip'])
|
||||
validate_archive(content, bootstrap.metadata['images'])
|
||||
bootstrap = replace(bootstrap, bundle_path=cached)
|
||||
except _FAILURES:
|
||||
pass
|
||||
return bootstrap
|
||||
@@ -1,11 +1,12 @@
|
||||
{
|
||||
"schema": 1,
|
||||
"version": "farm-assets-v1",
|
||||
"version": "catalogue-20261026-1",
|
||||
"sha256": "78d6e8e448fd5296b21d471b98ea7d58c81ef78a82b0e192a1ed041bb5fddad9",
|
||||
"size_bytes": 21264561,
|
||||
"catalogue_release": "4c88736c2e9b1b5eec4568e953152d59a6918242802da72a6eec1a615a17f18b",
|
||||
"sources": {
|
||||
"github": "https://raw.githubusercontent.com/Shu-Ying/farm_service/main/assets/assets.zip",
|
||||
"gitea": "https://gitea.diuse.work/Diuse/farm_service/raw/branch/main/assets/assets.zip",
|
||||
"farm_service": "https://farm.example/v1/assets/78d6e8e448fd5296b21d471b98ea7d58c81ef78a82b0e192a1ed041bb5fddad9/assets.zip"
|
||||
"github": "https://raw.githubusercontent.com/Shu-Ying/farm_service/main/catalogue/releases/4c88736c2e9b1b5eec4568e953152d59a6918242802da72a6eec1a615a17f18b/assets.zip",
|
||||
"gitea": "https://gitea.diuse.work/Diuse/farm_service/raw/branch/main/catalogue/releases/4c88736c2e9b1b5eec4568e953152d59a6918242802da72a6eec1a615a17f18b/assets.zip",
|
||||
"farm_service": "https://farm.example/v1/catalogue/releases/4c88736c2e9b1b5eec4568e953152d59a6918242802da72a6eec1a615a17f18b/assets.zip"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,171 @@
|
||||
"""校验同版本展示配置及ZIP图片,不将不可信路径解压到磁盘。"""
|
||||
|
||||
from io import BytesIO
|
||||
import math
|
||||
from pathlib import PurePosixPath
|
||||
from zipfile import ZipFile
|
||||
|
||||
from PIL import Image
|
||||
|
||||
from farm_core.catalogue_release import decode, digest
|
||||
|
||||
|
||||
def _numbers(values, length):
|
||||
if (type(values) is not list or len(values) != length
|
||||
or any(type(v) not in (int, float) or abs(v) > 10000000
|
||||
or not math.isfinite(v) for v in values)):
|
||||
raise ValueError('Invalid render coordinates')
|
||||
return values
|
||||
|
||||
|
||||
def _size(values):
|
||||
_numbers(values, 2)
|
||||
if (any(type(v) is not int or not 0 < v <= 8192 for v in values)
|
||||
or values[0] * values[1] > 32 * 1024 * 1024):
|
||||
raise ValueError('Invalid image dimensions')
|
||||
|
||||
|
||||
def _rect(values):
|
||||
_numbers(values, 4)
|
||||
if values[2] <= 0 or values[3] <= 0:
|
||||
raise ValueError('Invalid render rectangle')
|
||||
|
||||
|
||||
def safe_member(name):
|
||||
if (not isinstance(name, str) or not name or len(name) > 512
|
||||
or '\\' in name or ':' in name):
|
||||
raise ValueError('Invalid image path')
|
||||
path = PurePosixPath(name)
|
||||
if path.is_absolute() or any(p in ('', '.', '..') for p in name.split('/')):
|
||||
raise ValueError('Invalid image path')
|
||||
|
||||
|
||||
def validate_metadata(content, catalogue):
|
||||
value = decode(content)
|
||||
try:
|
||||
if type(value['schema']) is not int or value['schema'] != 1:
|
||||
raise ValueError('Unknown render metadata schema')
|
||||
_size(value['canvas'])
|
||||
_size(value['soil_size'])
|
||||
if len(value['positions']) != 30:
|
||||
raise ValueError('Expected 30 plot positions')
|
||||
for pos in value['positions']:
|
||||
_numbers([pos['x'], pos['y']], 2)
|
||||
images = value['images']
|
||||
if not isinstance(images, dict) or not 0 < len(images) <= 10000:
|
||||
raise ValueError('Invalid image manifest')
|
||||
for name, item in images.items():
|
||||
safe_member(name)
|
||||
_size(item['size'])
|
||||
if (not isinstance(item['sha256'], str)
|
||||
or len(item['sha256']) != 64
|
||||
or any(c not in '0123456789abcdef'
|
||||
for c in item['sha256'])):
|
||||
raise ValueError('Invalid image checksum')
|
||||
plants = value['plants']
|
||||
if set(plants) != {plant.name for plant in catalogue}:
|
||||
raise ValueError('Catalogue and render plants differ')
|
||||
for plant in catalogue:
|
||||
phases = plants[plant.name]['phase'].split(',')
|
||||
numbers = list(dict.fromkeys(int(v) for v in phases))
|
||||
if (len(numbers) < 2 or min(numbers) <= 0
|
||||
or numbers != sorted(numbers)):
|
||||
raise ValueError('Invalid render growth phases')
|
||||
placement = value['placement']
|
||||
if (type(placement['schema']) is not int
|
||||
or placement['schema'] != 4
|
||||
or placement['mode'] != 'visible_contact_center'):
|
||||
raise ValueError('Unknown crop placement')
|
||||
for key in ('common_scale', 'growth_scale', 'reference_soil_height',
|
||||
'reference_soil_width'):
|
||||
_numbers([placement[key]], 1)
|
||||
if placement[key] <= 0:
|
||||
raise ValueError('Invalid crop scale')
|
||||
_numbers([placement['contact_alpha_threshold']], 1)
|
||||
_numbers([placement['contact_band_fraction']], 1)
|
||||
if (not 0 <= placement['contact_alpha_threshold'] <= 255
|
||||
or not 0 < placement['contact_band_fraction'] <= 1):
|
||||
raise ValueError('Invalid ground contact rule')
|
||||
matrix = placement['scene_transform']
|
||||
if len(matrix) != 2:
|
||||
raise ValueError('Invalid scene transform')
|
||||
a, b = _numbers(matrix[0], 2)
|
||||
c, d = _numbers(matrix[1], 2)
|
||||
if abs(a*d-b*c) < 1e-8:
|
||||
raise ValueError('Degenerate scene transform')
|
||||
pest = value['pest']
|
||||
_size(pest['display_size'])
|
||||
if (len(pest['frames']) < 3
|
||||
or len(set(pest['frames'])) != len(pest['frames'])):
|
||||
raise ValueError('Invalid pest frames')
|
||||
for name in pest['frames']:
|
||||
if name not in images:
|
||||
raise ValueError('Missing pest image')
|
||||
if len(pest['positions']) < 3:
|
||||
raise ValueError('Missing pest positions')
|
||||
for pos in pest['positions']:
|
||||
_numbers(pos, 2)
|
||||
panel = value['panel']
|
||||
_size(panel['reference_canvas'])
|
||||
_rect(panel['avatar'])
|
||||
fields = {'name', 'experience', 'level', 'point', 'vip_point'}
|
||||
if {entry['field'] for entry in panel['fields']} != fields:
|
||||
raise ValueError('Invalid panel fields')
|
||||
for entry in panel['fields']:
|
||||
_rect(entry['rect'])
|
||||
_numbers(entry['color'], 3)
|
||||
if (any(not 0 <= v <= 255 for v in entry['color'])
|
||||
or entry['alignment'] not in (3, 4)):
|
||||
raise ValueError('Invalid panel text')
|
||||
for entry in panel['sprites']:
|
||||
_rect(entry['rect'])
|
||||
if entry['sprite'] not in images:
|
||||
raise ValueError('Missing panel sprite')
|
||||
if set(value['decorations']) != {
|
||||
'default', 'spring', 'summer', 'autumn', 'winter', 'bee'}:
|
||||
raise ValueError('Invalid decoration themes')
|
||||
for entries in value['decorations'].values():
|
||||
for entry in entries:
|
||||
_numbers(entry['position'], 2)
|
||||
_numbers(entry['size'], 2)
|
||||
if (min(entry['size']) <= 0
|
||||
or entry['layer'] not in ('background', 'foreground')
|
||||
or entry['sprite'] not in images):
|
||||
raise ValueError('Invalid decoration')
|
||||
for name in ('webgl/seed.png', 'webgl/wither.png',
|
||||
'background/ripe.png', 'soil/草土地.png',
|
||||
'soil/普通土地.png'):
|
||||
if name not in images:
|
||||
raise ValueError('Missing required farm sprite')
|
||||
return value
|
||||
except (KeyError, TypeError, IndexError, AttributeError) as error:
|
||||
raise ValueError('Invalid render metadata') from error
|
||||
|
||||
|
||||
def validate_archive(content, images=None):
|
||||
with ZipFile(BytesIO(content)) as archive:
|
||||
entries = archive.infolist()
|
||||
names = [entry.filename for entry in entries]
|
||||
expected = (set(images) if images is not None else {
|
||||
'soil/红土地.png', 'soil/黑土地.png', 'soil/金土地.png',
|
||||
})
|
||||
if (len(names) != len(set(names)) or set(names) != expected
|
||||
or sum(e.file_size for e in entries) > 256 * 1024 * 1024):
|
||||
raise ValueError('Unexpected release ZIP contents')
|
||||
for entry in entries:
|
||||
safe_member(entry.filename)
|
||||
if (entry.is_dir() or entry.flag_bits & 1
|
||||
or entry.file_size > 32 * 1024 * 1024
|
||||
or (entry.external_attr >> 16) & 0o170000 == 0o120000):
|
||||
raise ValueError('Invalid release ZIP member')
|
||||
data = archive.read(entry)
|
||||
expected_image = images[entry.filename] if images else None
|
||||
if (expected_image is not None
|
||||
and digest(data) != expected_image['sha256']):
|
||||
raise ValueError('Release image checksum differs')
|
||||
with Image.open(BytesIO(data)) as image:
|
||||
_size(list(image.size))
|
||||
if (expected_image is not None
|
||||
and list(image.size) != expected_image['size']):
|
||||
raise ValueError('Release image dimensions differ')
|
||||
image.verify()
|
||||
@@ -26,13 +26,14 @@ class FarmRenderer:
|
||||
def __init__(
|
||||
self, font_path, asset_cache_bytes=32 * 1024 * 1024,
|
||||
result_cache_bytes=8 * 1024 * 1024,
|
||||
bundle_path=None,
|
||||
bundle_path=None, metadata=None, soils_path=None,
|
||||
):
|
||||
self.font_path = Path(font_path).resolve()
|
||||
if not self.font_path.is_file():
|
||||
raise ValueError("Configure an existing Chinese font path")
|
||||
font(self.font_path, 16)
|
||||
self.assets = Assets(asset_cache_bytes, bundle_path=bundle_path)
|
||||
self.assets = Assets(asset_cache_bytes, bundle_path=bundle_path,
|
||||
metadata=metadata, soils_path=soils_path)
|
||||
self.results = ByteLRU(result_cache_bytes)
|
||||
|
||||
def scene(self, identity, snapshot, now):
|
||||
|
||||
@@ -46,12 +46,13 @@ class BundleTests(unittest.TestCase):
|
||||
bundle.ensure_assets(self.root / 'gitea', 'Gitea')
|
||||
self.assertEqual(fetch.call_args.args[0],
|
||||
'https://gitea.diuse.work/Diuse/farm_service/'
|
||||
'raw/branch/main/assets/assets.zip')
|
||||
'raw/branch/main/catalogue/releases/'
|
||||
+ self.expected['catalogue_release'] + '/assets.zip')
|
||||
bundle.ensure_assets(self.root / 'service', 'farm_service',
|
||||
'https://farm.example/prefix/')
|
||||
self.assertEqual(fetch.call_args.args[0],
|
||||
'https://farm.example/prefix/v1/assets/'
|
||||
+ self.expected['sha256'] + '/assets.zip')
|
||||
'https://farm.example/prefix/v1/catalogue/releases/'
|
||||
+ self.expected['catalogue_release'] + '/assets.zip')
|
||||
|
||||
def test_failed_download_leaves_no_partial_and_wrong_hash_is_rejected(self):
|
||||
def corrupt(url, destination, limit):
|
||||
|
||||
@@ -146,7 +146,12 @@ class AstrBotPackageTests(unittest.IsolatedAsyncioTestCase):
|
||||
Path(choose_font()).read_bytes())
|
||||
(fonts / 'OFL-Xiaolai.txt').write_bytes((ROOT /
|
||||
'src/farm_render/data/OFL-Xiaolai.txt').read_bytes())
|
||||
await plugin.initialize()
|
||||
with patch(
|
||||
package.__name__ +
|
||||
'._vendor.farm_render.catalogue_bundle._fetch',
|
||||
side_effect=OSError('offline fixture'),
|
||||
):
|
||||
await plugin.initialize()
|
||||
self.assertEqual(set(commands), set(main.COMMANDS))
|
||||
|
||||
class Event:
|
||||
|
||||
@@ -0,0 +1,195 @@
|
||||
import copy
|
||||
import hashlib
|
||||
import json
|
||||
from io import BytesIO
|
||||
from zipfile import ZipFile
|
||||
from pathlib import Path
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from farm_core.catalogue_release import make_manifest, parse_manifest
|
||||
from farm_render.catalogue_bundle import load_catalogue_release
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
|
||||
def release(price=21, extra=False):
|
||||
plants = json.loads((ROOT / 'src/farm_core/data/plants.json').read_bytes())
|
||||
plants.pop('source_sha256', None)
|
||||
plants['plants'][0]['price'] = price
|
||||
metadata = json.loads(
|
||||
(ROOT / 'src/farm_render/data/metadata.json').read_bytes()
|
||||
)
|
||||
if extra:
|
||||
row = copy.deepcopy(plants['plants'][0])
|
||||
row.update(id=9001, name='更新测试作物')
|
||||
plants['plants'].append(row)
|
||||
metadata['plants'][row['name']] = copy.deepcopy(
|
||||
metadata['plants'][plants['plants'][0]['name']]
|
||||
)
|
||||
files = {
|
||||
'plants.json': json.dumps(plants, ensure_ascii=False).encode(),
|
||||
'metadata.json': json.dumps(metadata, ensure_ascii=False).encode(),
|
||||
**{name: (ROOT / 'src/farm_render/data' / name).read_bytes()
|
||||
for name in ('assets.zip', 'land-soils.zip')},
|
||||
}
|
||||
return make_manifest(files, 'test-v1'), files
|
||||
|
||||
|
||||
class CatalogueReleaseTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.root = Path(self.temp.name)
|
||||
self.manifest, self.files = release(price=99)
|
||||
|
||||
def tearDown(self):
|
||||
self.temp.cleanup()
|
||||
|
||||
def fetch(self, url, limit):
|
||||
if url.endswith('/manifest.json'):
|
||||
data = json.dumps(self.manifest).encode()
|
||||
else:
|
||||
self.assertIn('/releases/' + self.manifest['release'] + '/', url)
|
||||
data = self.files[url.rsplit('/', 1)[-1]]
|
||||
self.assertLessEqual(len(data), limit)
|
||||
return data
|
||||
|
||||
def load(self, **kwargs):
|
||||
return load_catalogue_release(self.root, **kwargs)
|
||||
|
||||
def test_new_catalogue_and_metadata_activate_as_one_version(self):
|
||||
with patch('farm_render.catalogue_bundle._fetch',
|
||||
side_effect=self.fetch):
|
||||
result = self.load()
|
||||
self.assertEqual(result.catalogue.get_by_id(1).price, 99)
|
||||
self.assertEqual(result.origin, 'remote')
|
||||
self.assertTrue(result.bundle_path.is_file())
|
||||
self.assertTrue(result.soils_path.is_file())
|
||||
current = json.loads((self.root / 'catalogue/current.json').read_bytes())
|
||||
self.assertEqual(current['release'], self.manifest['release'])
|
||||
|
||||
def test_404_reuses_verified_current(self):
|
||||
with patch('farm_render.catalogue_bundle._fetch',
|
||||
side_effect=self.fetch):
|
||||
self.load()
|
||||
with patch('farm_render.catalogue_bundle._fetch',
|
||||
side_effect=OSError('404')):
|
||||
result = self.load()
|
||||
self.assertEqual(result.catalogue.get_by_id(1).price, 99)
|
||||
self.assertEqual(result.origin, 'cache')
|
||||
|
||||
def test_wrong_bytes_do_not_replace_current(self):
|
||||
with patch('farm_render.catalogue_bundle._fetch',
|
||||
side_effect=self.fetch):
|
||||
self.load()
|
||||
before = (self.root / 'catalogue/current.json').read_bytes()
|
||||
self.manifest, self.files = release(price=100)
|
||||
self.files['plants.json'] = b'broken'
|
||||
with patch('farm_render.catalogue_bundle._fetch',
|
||||
side_effect=self.fetch):
|
||||
result = self.load()
|
||||
self.assertEqual(result.catalogue.get_by_id(1).price, 99)
|
||||
self.assertEqual((self.root / 'catalogue/current.json').read_bytes(),
|
||||
before)
|
||||
|
||||
def test_first_offline_start_uses_bootstrap_without_images(self):
|
||||
with patch('farm_render.catalogue_bundle._fetch',
|
||||
side_effect=OSError('offline')):
|
||||
result = self.load()
|
||||
self.assertEqual(result.origin, 'bootstrap')
|
||||
self.assertEqual(len(result.catalogue), 202)
|
||||
self.assertFalse((self.root / 'catalogue/current.json').exists())
|
||||
|
||||
def test_cache_loss_with_unknown_player_id_refuses_start(self):
|
||||
with patch('farm_render.catalogue_bundle._fetch',
|
||||
side_effect=OSError('offline')):
|
||||
with self.assertRaisesRegex(ValueError, 'catalogue.*ID'):
|
||||
self.load(required_ids={9001})
|
||||
|
||||
def test_incompatible_downgrade_keeps_added_crop(self):
|
||||
self.manifest, self.files = release(extra=True)
|
||||
with patch('farm_render.catalogue_bundle._fetch',
|
||||
side_effect=self.fetch):
|
||||
self.load()
|
||||
self.manifest, self.files = release()
|
||||
with patch('farm_render.catalogue_bundle._fetch',
|
||||
side_effect=self.fetch):
|
||||
result = self.load()
|
||||
self.assertEqual(result.catalogue.get_by_id(9001).name, '更新测试作物')
|
||||
self.assertEqual(result.origin, 'cache')
|
||||
|
||||
def test_manifest_rejects_bool_unknown_schema_paths_and_fingerprint(self):
|
||||
for key, value in [('schema', True), ('schema', 2),
|
||||
('release', '0' * 64)]:
|
||||
bad = copy.deepcopy(self.manifest)
|
||||
bad[key] = value
|
||||
with self.assertRaises(ValueError):
|
||||
parse_manifest(json.dumps(bad).encode())
|
||||
bad = copy.deepcopy(self.manifest)
|
||||
bad['files']['../file'] = bad['files'].pop('assets.zip')
|
||||
with self.assertRaises(ValueError):
|
||||
parse_manifest(json.dumps(bad).encode())
|
||||
|
||||
def test_selected_service_prefix_is_respected(self):
|
||||
seen = []
|
||||
def fetch(url, limit):
|
||||
seen.append(url)
|
||||
return self.fetch(url, limit)
|
||||
with patch('farm_render.catalogue_bundle._fetch', side_effect=fetch):
|
||||
self.load(source='farm_service',
|
||||
service_url='https://farm.test/prefix')
|
||||
self.assertTrue(all(
|
||||
url.startswith('https://farm.test/prefix/v1/catalogue/')
|
||||
for url in seen
|
||||
))
|
||||
|
||||
def test_current_matching_remote_still_checks_player_ids(self):
|
||||
with patch('farm_render.catalogue_bundle._fetch',
|
||||
side_effect=self.fetch):
|
||||
self.load()
|
||||
with self.assertRaisesRegex(ValueError, 'catalogue.*ID'):
|
||||
self.load(required_ids={9001})
|
||||
|
||||
def test_consistent_but_invalid_metadata_keeps_current(self):
|
||||
with patch('farm_render.catalogue_bundle._fetch',
|
||||
side_effect=self.fetch):
|
||||
self.load()
|
||||
self.files['metadata.json'] = b'{"schema":1,"canvas":[0,0]}'
|
||||
self.manifest = make_manifest(self.files, 'bad-layout')
|
||||
result = self.load()
|
||||
self.assertEqual(result.origin, 'cache')
|
||||
self.assertEqual(result.metadata['canvas'], [3072, 1872])
|
||||
|
||||
def test_huge_coordinate_keeps_verified_current(self):
|
||||
with patch('farm_render.catalogue_bundle._fetch',
|
||||
side_effect=self.fetch):
|
||||
self.load()
|
||||
metadata = json.loads(self.files['metadata.json'])
|
||||
metadata['positions'][0]['x'] = 10 ** 500
|
||||
self.files['metadata.json'] = json.dumps(metadata).encode()
|
||||
self.manifest = make_manifest(self.files, 'huge-coordinate')
|
||||
result = self.load()
|
||||
self.assertEqual(result.origin, 'cache')
|
||||
self.assertLess(result.metadata['positions'][0]['x'], 10000000)
|
||||
|
||||
def test_removing_old_crop_image_is_rejected_even_with_valid_hashes(self):
|
||||
with patch('farm_render.catalogue_bundle._fetch',
|
||||
side_effect=self.fetch):
|
||||
self.load()
|
||||
metadata = json.loads(self.files['metadata.json'])
|
||||
name = next(key for key in metadata['images']
|
||||
if key.startswith('plant/'))
|
||||
metadata['images'].pop(name)
|
||||
output = BytesIO()
|
||||
with ZipFile(BytesIO(self.files['assets.zip'])) as old, \
|
||||
ZipFile(output, 'w') as new:
|
||||
for item in old.infolist():
|
||||
if item.filename != name:
|
||||
new.writestr(item, old.read(item))
|
||||
self.files['assets.zip'] = output.getvalue()
|
||||
self.files['metadata.json'] = json.dumps(metadata).encode()
|
||||
self.manifest = make_manifest(self.files, 'missing-image')
|
||||
result = self.load()
|
||||
self.assertEqual(result.origin, 'cache')
|
||||
self.assertIn(name, result.metadata['images'])
|
||||
@@ -0,0 +1,34 @@
|
||||
import json
|
||||
from contextlib import closing
|
||||
from pathlib import Path
|
||||
import sqlite3
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
from farm_core import SQLiteStore
|
||||
|
||||
|
||||
class CatalogueStoreTests(unittest.IsolatedAsyncioTestCase):
|
||||
async def test_missing_store_is_not_created_by_catalogue_check(self):
|
||||
with tempfile.TemporaryDirectory() as temporary:
|
||||
path = Path(temporary) / 'not-created.sqlite3'
|
||||
store = SQLiteStore(path)
|
||||
self.assertEqual(await store.catalogue_ids(), set())
|
||||
self.assertFalse(path.exists())
|
||||
|
||||
async def test_existing_ids_read_without_changing_database(self):
|
||||
with tempfile.TemporaryDirectory() as temporary:
|
||||
path = Path(temporary) / 'farm.sqlite3'
|
||||
store = SQLiteStore(path)
|
||||
await store.initialize()
|
||||
await store.close()
|
||||
with closing(sqlite3.connect(path)) as db, db:
|
||||
db.execute('INSERT INTO accounts(identity,name) VALUES (?,?)',
|
||||
('fixture', 'synthetic'))
|
||||
db.execute('INSERT INTO inventory(identity,plant_id,name,seeds)'
|
||||
' VALUES (?,?,?,?)', ('fixture', 9001, 'test', 1))
|
||||
db.execute('INSERT INTO plots(identity,slot,plant) VALUES (?,?,?)',
|
||||
('fixture', 1, json.dumps({'plant_id': 9002})))
|
||||
before = path.read_bytes()
|
||||
self.assertEqual(await store.catalogue_ids(), {9001, 9002})
|
||||
self.assertEqual(path.read_bytes(), before)
|
||||
@@ -11,6 +11,8 @@ from export_render_assets import export
|
||||
class RenderExportTests(unittest.TestCase):
|
||||
def test_export_preserves_published_land_numbering_and_calibration(self):
|
||||
_, metadata = export(ROOT)
|
||||
stored = (ROOT / "src/farm_render/data/metadata.json").read_bytes()
|
||||
self.assertEqual(json.loads(metadata), json.loads(stored))
|
||||
self.assertEqual(metadata, stored)
|
||||
data = json.loads(metadata)
|
||||
self.assertEqual(len(data['plants']), 202)
|
||||
self.assertEqual(len(data['positions']), 30)
|
||||
self.assertEqual(data['placement']['mode'], 'visible_contact_center')
|
||||
self.assertEqual(metadata, export(ROOT)[1])
|
||||
|
||||
@@ -27,6 +27,12 @@ class ZhenxunRuntimeTests(unittest.IsolatedAsyncioTestCase):
|
||||
module.__path__ = [str(self.root / "built/repository")]
|
||||
sys.modules[self.package] = module
|
||||
self.module = importlib.import_module(self.package + ".runtime")
|
||||
offline = patch(
|
||||
self.package + '._vendor.farm_render.catalogue_bundle._fetch',
|
||||
side_effect=OSError('offline fixture'),
|
||||
)
|
||||
offline.start()
|
||||
self.addCleanup(offline.stop)
|
||||
self.runtime = self.module.Runtime(self.root / "data", choose_font())
|
||||
self.seed_assets(self.runtime)
|
||||
await self.runtime.start()
|
||||
@@ -78,6 +84,133 @@ class ZhenxunRuntimeTests(unittest.IsolatedAsyncioTestCase):
|
||||
for action in ('seeds', 'crops'):
|
||||
self.assertIsInstance(await self.call(action, '1'), bytes)
|
||||
|
||||
async def test_catalogue_is_loaded_once_before_store_initialization(self):
|
||||
import json
|
||||
from unittest.mock import AsyncMock
|
||||
|
||||
await self.call('register')
|
||||
await self.call('buy', '胡萝卜 1', message='seed')
|
||||
await self.call('sow', '胡萝卜 1', message='sow')
|
||||
before = await self.runtime.service.snapshot(self.runtime_identity())
|
||||
await self.runtime.close()
|
||||
bundle = importlib.import_module(
|
||||
self.package + '._vendor.farm_render.catalogue_bundle',
|
||||
)
|
||||
original = bundle._bootstrap()
|
||||
document = json.loads((ROOT / 'src/farm_core/data/plants.json')
|
||||
.read_bytes())
|
||||
document['plants'][0]['price'] = 99
|
||||
document['plants'][0]['harvest'] = 999
|
||||
updated = replace(original, catalogue=self.module.Catalogue.loads(
|
||||
json.dumps(document),
|
||||
))
|
||||
self.runtime = self.module.Runtime(self.root / 'data', choose_font())
|
||||
self.seed_assets(self.runtime)
|
||||
with patch.object(self.module, 'load_catalogue_release',
|
||||
return_value=updated) as load:
|
||||
await asyncio.gather(self.runtime.start(), self.runtime.start())
|
||||
await self.runtime.start()
|
||||
self.assertEqual(load.call_count, 1)
|
||||
self.assertEqual(self.runtime.service.catalogue.get_by_id(1).price, 99)
|
||||
after = await self.runtime.service.snapshot(self.runtime_identity())
|
||||
self.assertEqual(after.plots[0].plant, before.plots[0].plant)
|
||||
self.assertNotEqual(after.plots[0].plant.harvest, 999)
|
||||
|
||||
async def test_close_during_catalogue_load_does_not_start_service(self):
|
||||
import threading
|
||||
|
||||
await self.runtime.close()
|
||||
bundle = importlib.import_module(
|
||||
self.package + '._vendor.farm_render.catalogue_bundle',
|
||||
)
|
||||
snapshot = bundle._bootstrap()
|
||||
entered, release = threading.Event(), threading.Event()
|
||||
def slow(*args):
|
||||
entered.set()
|
||||
if not release.wait(5):
|
||||
raise TimeoutError('fixture did not release')
|
||||
return snapshot
|
||||
self.runtime = self.module.Runtime(self.root / 'other', choose_font())
|
||||
with patch.object(self.module, 'load_catalogue_release',
|
||||
side_effect=slow):
|
||||
starting = asyncio.create_task(self.runtime.start())
|
||||
await asyncio.wait_for(asyncio.to_thread(entered.wait), 2)
|
||||
closing = asyncio.create_task(self.runtime.close())
|
||||
await asyncio.sleep(0)
|
||||
release.set()
|
||||
await asyncio.gather(starting, closing)
|
||||
self.assertFalse(self.runtime.started)
|
||||
self.assertFalse((self.root / 'other/farm.sqlite3').exists())
|
||||
|
||||
async def test_close_during_id_read_prevents_later_start(self):
|
||||
await self.runtime.close()
|
||||
self.runtime = self.module.Runtime(self.root / 'other', choose_font())
|
||||
self.seed_assets(self.runtime)
|
||||
entered, release = asyncio.Event(), asyncio.Event()
|
||||
async def slow_ids():
|
||||
entered.set()
|
||||
await release.wait()
|
||||
return set()
|
||||
with patch.object(self.runtime.store, 'catalogue_ids',
|
||||
side_effect=slow_ids):
|
||||
starting = asyncio.create_task(self.runtime.start())
|
||||
await entered.wait()
|
||||
await self.runtime.close()
|
||||
release.set()
|
||||
await starting
|
||||
self.assertFalse(self.runtime.started)
|
||||
self.assertFalse((self.root / 'other/farm.sqlite3').exists())
|
||||
|
||||
async def test_cancelled_start_drains_catalogue_download(self):
|
||||
import threading
|
||||
|
||||
await self.runtime.close()
|
||||
bundle = importlib.import_module(
|
||||
self.package + '._vendor.farm_render.catalogue_bundle',
|
||||
)
|
||||
snapshot = bundle._bootstrap()
|
||||
entered, release, finished = (threading.Event() for _ in range(3))
|
||||
def slow(*args):
|
||||
entered.set()
|
||||
if not release.wait(5):
|
||||
raise TimeoutError('fixture did not release')
|
||||
finished.set()
|
||||
return snapshot
|
||||
self.runtime = self.module.Runtime(self.root / 'other', choose_font())
|
||||
with patch.object(self.module, 'load_catalogue_release',
|
||||
side_effect=slow):
|
||||
starting = asyncio.create_task(self.runtime.start())
|
||||
await asyncio.wait_for(asyncio.to_thread(entered.wait), 2)
|
||||
starting.cancel()
|
||||
await asyncio.sleep(0.01)
|
||||
self.assertFalse(starting.done())
|
||||
release.set()
|
||||
with self.assertRaises(asyncio.CancelledError):
|
||||
await starting
|
||||
self.assertTrue(finished.is_set())
|
||||
self.assertIsNone(self.runtime.log_handler)
|
||||
self.assertFalse(self.runtime.started)
|
||||
|
||||
async def test_close_during_initialization_prevents_later_start(self):
|
||||
await self.runtime.close()
|
||||
self.runtime = self.module.Runtime(self.root / 'other', choose_font())
|
||||
self.seed_assets(self.runtime)
|
||||
entered, release = asyncio.Event(), asyncio.Event()
|
||||
initialize = self.runtime.store.initialize
|
||||
async def slow_initialize(**kwargs):
|
||||
await initialize(**kwargs)
|
||||
entered.set()
|
||||
await release.wait()
|
||||
with patch.object(self.runtime.store, 'initialize',
|
||||
side_effect=slow_initialize):
|
||||
starting = asyncio.create_task(self.runtime.start())
|
||||
await entered.wait()
|
||||
await self.runtime.close()
|
||||
release.set()
|
||||
await starting
|
||||
self.assertFalse(self.runtime.started)
|
||||
self.assertIsNone(self.runtime.rendering)
|
||||
|
||||
async def test_land_upgrade_command_conditions_charge_and_replay(self):
|
||||
self.assertEqual(self.module.COMMANDS['土地升级'], 'upgrade')
|
||||
await self.call('register')
|
||||
@@ -577,6 +710,8 @@ class ZhenxunRuntimeTests(unittest.IsolatedAsyncioTestCase):
|
||||
for resource in ('assets', 'font'):
|
||||
other = self.module.Runtime(self.root / ('stop-' + resource))
|
||||
self.seed_assets(other)
|
||||
if resource == 'assets':
|
||||
next((other.data_dir / 'assets').glob('*/assets.zip')).unlink()
|
||||
try:
|
||||
with patch.object(self.module, 'ensure_' + resource,
|
||||
side_effect=TimeoutError()) as fetch:
|
||||
@@ -598,6 +733,7 @@ class ZhenxunRuntimeTests(unittest.IsolatedAsyncioTestCase):
|
||||
await asyncio.sleep(0)
|
||||
self.assertEqual(fetch.call_count, 1)
|
||||
await other.close()
|
||||
self.seed_assets(other)
|
||||
await other.start()
|
||||
self.assertIsNotNone(other.rendering)
|
||||
finally:
|
||||
@@ -614,6 +750,8 @@ class ZhenxunRuntimeTests(unittest.IsolatedAsyncioTestCase):
|
||||
):
|
||||
other = self.module.Runtime(self.root / ('failed-' + resource))
|
||||
self.seed_assets(other)
|
||||
if resource == 'assets':
|
||||
next((other.data_dir / 'assets').glob('*/assets.zip')).unlink()
|
||||
try:
|
||||
with patch.object(self.module, 'ensure_' + resource,
|
||||
side_effect=failure), \
|
||||
|
||||
+1
-1
@@ -1 +1 @@
|
||||
2.0.2
|
||||
2.0.3
|
||||
Reference in New Issue
Block a user