mirror of
https://github.com/Shu-Ying/main_plugin_farm.git
synced 2026-11-04 03:30:16 +08:00
fixed # 修复迁移数据库失败的BUG
This commit is contained in:
+29
-1
@@ -33,7 +33,8 @@ FARM_SERVICE_TOKEN,未把原值复制到新仓库。
|
|||||||
|
|
||||||
## 真实用户数据迁移建议
|
## 真实用户数据迁移建议
|
||||||
|
|
||||||
当前没有读写任何真实玩家库,以下尚未实施:
|
当前没有读写任何真实玩家库。以下是真实部署时的建议;已实现的自动迁移
|
||||||
|
范围见下一节,不能将隔离测试等同于真实玩家数据验收:
|
||||||
|
|
||||||
1. 盘点实际部署版本、schema、来源平台、玩家数量和货币含义。
|
1. 盘点实际部署版本、schema、来源平台、玩家数量和货币含义。
|
||||||
2. 停写并备份,记录 schema、文件哈希和备份位置;备份不入 Git。
|
2. 停写并备份,记录 schema、文件哈希和备份位置;备份不入 Git。
|
||||||
@@ -50,6 +51,33 @@ FARM_SERVICE_TOKEN,未把原值复制到新仓库。
|
|||||||
是否保留已有资产、重置、补偿或人工对账必须由用户明确决定,
|
是否保留已有资产、重置、补偿或人工对账必须由用户明确决定,
|
||||||
不在迁移脚本中暗自纠正或清零。
|
不在迁移脚本中暗自纠正或清零。
|
||||||
|
|
||||||
|
## 真寻旧版 farm.db 自动迁移
|
||||||
|
|
||||||
|
新版默认使用 `data/farm_db/farm.sqlite3`,只读同目录的旧版 `farm.db`。
|
||||||
|
首次收到用户农场指令时,按旧裸 UID 查找数据并转换;没有旧账号则沿用
|
||||||
|
正常注册流程。旧程序应停止写入,切换前自行备份,旧文件不会修改或删除。
|
||||||
|
|
||||||
|
- 身份绑定:使用首次请求的 deployment、机器人 ID、平台、用户 ID。
|
||||||
|
导入记录和资产在同一 SQLite 事务提交,同 UID 在整个目标库只导入一次。
|
||||||
|
再次请求不重置余额,重启后仍保持绑定;其他机器人或平台请求会被拒绝。
|
||||||
|
- 账户:保留名称、point、vipPoint、exp;不额外发放注册初始资产。
|
||||||
|
初始资产写入新版 ledger,并关联 `legacy_import` 操作。
|
||||||
|
- 仓库:按名称唯一匹配内置作物 ID,合并种子与产物,保留数量与锁定。
|
||||||
|
同名歧义、未知作物、重复行、非法数值都会回滚,不猜测或清零。
|
||||||
|
- 土地:按 soil 创建已解锁地块,缺失的行补为空地;保留作物、成熟时间、
|
||||||
|
收获季数与枯萎状态,土地等级映射为原有产量/经验加成。
|
||||||
|
旧虫害标志映射为一只虫;当前季保持旧成熟时间,不追溯套用新版生虫时钟,
|
||||||
|
下一季开始使用新版生长规则。
|
||||||
|
- 保留:所有含 uid 字段的旧表中该用户的原始行,存入
|
||||||
|
`operations.result` 的 `legacy_rows`。道具、签到、偷菜、杂草、缺水、
|
||||||
|
施肥及土地等级原值等新版尚无完整对应玩法的状态可在此核查;这些状态
|
||||||
|
不宣称已恢复为可用功能。额外表列在 `unsupported_tables`,日志记录表行数。
|
||||||
|
- 冲突:新账号已经存在时不覆盖、不累加;迁移失败不注册空账号。
|
||||||
|
检查服务器日志,修复副本并核对后再重试;原始旧库和已提交的新账号均保留。
|
||||||
|
|
||||||
|
`farm_preview` 目录与未发布的 `farm-preview.sqlite3` 不在此迁移范围。
|
||||||
|
本实现使用构造的旧版结构做隔离测试,尚未验证用户服务器上的真实数据库。
|
||||||
|
|
||||||
## 首次发布仓库接管
|
## 首次发布仓库接管
|
||||||
|
|
||||||
当前本地 Beta 基线不等于三个发布仓库 main。
|
当前本地 Beta 基线不等于三个发布仓库 main。
|
||||||
|
|||||||
@@ -28,16 +28,16 @@
|
|||||||
|
|
||||||
## 数据目录
|
## 数据目录
|
||||||
|
|
||||||
默认路径改为宿主 `DATA_PATH / farm_db`。为保持数据连续,数据库文件名
|
默认路径为宿主 `DATA_PATH / farm_db`,新版数据库名为 `farm.sqlite3`。
|
||||||
`farm-preview.sqlite3`、日志文件名和 deployment 标识不变。
|
日志文件名和 deployment 标识保持不变。不再自动移动 `farm_preview`
|
||||||
|
目录,也不再因两个目录同时存在而阻止启动。
|
||||||
|
|
||||||
更新插件后下次启动时,若默认旧目录 `farm_preview` 中存在本项目 SQLite,
|
同目录旧版 `farm.db` 按用户在首次使用指令时迁移。用户永久绑定首次请求
|
||||||
且 `farm_db` 不存在,会把整个旧目录重命名为 `farm_db`,同时保留日志和
|
的 deployment、机器人 ID 和平台,不跨身份重复导入;转换规则、失败处理
|
||||||
SQLite 附属文件。迁移在打开数据库前进行,成功后记录迁移日志。
|
和不支持字段的保留方式见 [旧数据迁移](05-migration.md)。
|
||||||
|
|
||||||
若两个目录同时存在,启动失败并提示明确选择数据目录,禁止覆盖或默默
|
原有显式 `farm_preview_data_dir` 配置继续生效。仅从选定目录读取旧版
|
||||||
创建空账户。文件被占用导致重命名失败时同样终止启动;应停止旧实例再启动。
|
`farm.db`;不会搜索其他目录。未发布预览版的 `farm-preview.sqlite3`
|
||||||
原有显式 `farm_preview_data_dir` 配置继续生效;该配置用于自定义路径,
|
不自动改名或导入,预览数据由开发者手动处理。
|
||||||
不会被自动移动。若显式配置仍指向 `farm_preview`,需自行更改该路径。
|
|
||||||
|
|
||||||
本轮只修改开发源并进行隔离测试;没有移动正在使用的宿主数据或重启机器人。
|
本轮只修改开发源并进行隔离测试;没有移动正在使用的宿主数据或重启机器人。
|
||||||
|
|||||||
@@ -2,7 +2,8 @@
|
|||||||
|
|
||||||
发布目标:https://github.com/Shu-Ying/zhenxun_plugin_farm.git
|
发布目标:https://github.com/Shu-Ying/zhenxun_plugin_farm.git
|
||||||
原指令、独立预览数据目录、身份和事件幂等、启动/关闭及发图接入。
|
原指令、独立预览数据目录、身份和事件幂等、启动/关闭及发图接入。
|
||||||
不接入真寻金币兑换,不迁移旧数据。核心/渲染由主仓库白名单打包,
|
不接入真寻金币兑换;旧版 farm.db 首次使用时按用户迁移到 farm.sqlite3。
|
||||||
|
核心/渲染由主仓库白名单打包,
|
||||||
生成时修正为插件私有相对导入,本目录本身不是完整安装包。
|
生成时修正为插件私有相对导入,本目录本身不是完整安装包。
|
||||||
使用 scripts/build_zhenxun.py 生成ZIP,详见 docs/11-zhenxun-preview.md。
|
使用 scripts/build_zhenxun.py 生成ZIP,详见 docs/11-zhenxun-preview.md。
|
||||||
本轮没有拉取、推送或操作真实机器人。
|
本轮没有拉取、推送或操作真实机器人。
|
||||||
|
|||||||
@@ -15,12 +15,12 @@ from zhenxun.utils.message import MessageUtils
|
|||||||
from zhenxun.utils.platform import PlatformUtils
|
from zhenxun.utils.platform import PlatformUtils
|
||||||
|
|
||||||
from .runtime import COMMANDS, Runtime
|
from .runtime import COMMANDS, Runtime
|
||||||
from .data_paths import data_directories
|
from .data_paths import data_directory
|
||||||
from .config_migration import migrate_config
|
from .config_migration import migrate_config
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
driver = get_driver()
|
driver = get_driver()
|
||||||
data_dir, legacy_data_dir = data_directories(
|
data_dir = data_directory(
|
||||||
DATA_PATH, getattr(driver.config, 'farm_preview_data_dir', None),
|
DATA_PATH, getattr(driver.config, 'farm_preview_data_dir', None),
|
||||||
)
|
)
|
||||||
runtime = Runtime(
|
runtime = Runtime(
|
||||||
@@ -29,7 +29,6 @@ runtime = Runtime(
|
|||||||
getattr(driver.config, "farm_preview_deployment", "zhenxun-preview"),
|
getattr(driver.config, "farm_preview_deployment", "zhenxun-preview"),
|
||||||
str(getattr(driver.config, "farm_preview_image_format", "AUTO")).upper(),
|
str(getattr(driver.config, "farm_preview_image_format", "AUTO")).upper(),
|
||||||
font_dir=FONT_PATH,
|
font_dir=FONT_PATH,
|
||||||
legacy_data_dir=legacy_data_dir,
|
|
||||||
)
|
)
|
||||||
async def start_runtime():
|
async def start_runtime():
|
||||||
from zhenxun.configs.config import Config
|
from zhenxun.configs.config import Config
|
||||||
|
|||||||
@@ -1,30 +1,9 @@
|
|||||||
"""农场默认目录与旧预览目录的无覆盖迁移。"""
|
"""农场数据目录选择。"""
|
||||||
|
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
def data_directories(base, configured=None):
|
def data_directory(base, configured=None):
|
||||||
if configured is not None:
|
if configured is not None:
|
||||||
return Path(configured), None
|
return Path(configured)
|
||||||
return Path(base) / 'farm_db', Path(base) / 'farm_preview'
|
return Path(base) / 'farm_db'
|
||||||
|
|
||||||
|
|
||||||
def prepare_directory(target, legacy=None):
|
|
||||||
target = Path(target).absolute()
|
|
||||||
if legacy is not None:
|
|
||||||
legacy = Path(legacy).absolute()
|
|
||||||
if (target.name != 'farm_db' or legacy.name != 'farm_preview'
|
|
||||||
or target.parent.resolve() != legacy.parent.resolve()):
|
|
||||||
raise ValueError('Farm migration requires default sibling paths')
|
|
||||||
if target.is_symlink() or legacy.is_symlink():
|
|
||||||
raise ValueError('Farm migration does not follow directory links')
|
|
||||||
if (legacy / 'farm-preview.sqlite3').is_file():
|
|
||||||
if target.exists():
|
|
||||||
raise FileExistsError(
|
|
||||||
'farm_db already exists beside farm_preview; '
|
|
||||||
'select farm_preview_data_dir explicitly before startup'
|
|
||||||
)
|
|
||||||
legacy.rename(target)
|
|
||||||
return True
|
|
||||||
target.mkdir(parents=True, exist_ok=True)
|
|
||||||
return False
|
|
||||||
|
|||||||
@@ -13,7 +13,6 @@ from ._vendor.farm_core.presentation import receipt_text, snapshot_text
|
|||||||
from ._vendor.farm_render import FarmRenderer, RenderOptions, RenderService
|
from ._vendor.farm_render import FarmRenderer, RenderOptions, RenderService
|
||||||
from ._vendor.farm_render.seasons import season_for
|
from ._vendor.farm_render.seasons import season_for
|
||||||
from ._vendor.farm_render.shop import shop_page
|
from ._vendor.farm_render.shop import shop_page
|
||||||
from .data_paths import prepare_directory
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
COMMANDS = {
|
COMMANDS = {
|
||||||
@@ -36,9 +35,13 @@ HELP = (
|
|||||||
"每只虫使生长速度降低5%;成熟时每只虫使本季减产5%。\n"
|
"每只虫使生长速度降低5%;成熟时每只虫使本季减产5%。\n"
|
||||||
"成熟后除虫不能恢复本季减产;生长期间可发送:农场除虫。\n"
|
"成熟后除虫不能恢复本季减产;生长期间可发送:农场除虫。\n"
|
||||||
"我的农场按北京时间自动切换春夏秋冬背景;高清使用相同季节。\n"
|
"我的农场按北京时间自动切换春夏秋冬背景;高清使用相同季节。\n"
|
||||||
"本版使用独立预览数据库,不会读取旧版资产。"
|
"旧版农场数据在首次使用时迁移,绑定当前机器人和平台。"
|
||||||
)
|
)
|
||||||
ERRORS = {
|
ERRORS = {
|
||||||
|
"legacy_identity_bound":"旧农场已绑定其他机器人或平台,请联系管理员。",
|
||||||
|
"legacy_account_conflict":"新旧农场账号冲突,未覆盖,请联系管理员。",
|
||||||
|
"legacy_schema_unsupported":"旧农场数据库结构不兼容,请联系管理员。",
|
||||||
|
"legacy_data_invalid":"旧农场数据异常,迁移已回滚,请联系管理员。",
|
||||||
"not_registered":"请先发送:开通农场",
|
"not_registered":"请先发送:开通农场",
|
||||||
"unknown_plant":"没有这种作物,请查看种子商店。",
|
"unknown_plant":"没有这种作物,请查看种子商店。",
|
||||||
"ambiguous_plant":"存在同名作物,请使用商店或仓库中的 #编号。",
|
"ambiguous_plant":"存在同名作物,请使用商店或仓库中的 #编号。",
|
||||||
@@ -69,23 +72,23 @@ def operation_id(bot_id, platform, user_id, message_id, action):
|
|||||||
|
|
||||||
class Runtime:
|
class Runtime:
|
||||||
def __init__(self, data_dir, font_path=None, deployment="zhenxun-preview",
|
def __init__(self, data_dir, font_path=None, deployment="zhenxun-preview",
|
||||||
image_format="AUTO", *, font_dir=None, legacy_data_dir=None):
|
image_format="AUTO", *, font_dir=None):
|
||||||
self.data_dir = Path(data_dir).resolve()
|
self.data_dir = Path(data_dir).resolve()
|
||||||
self.legacy_data_dir = legacy_data_dir
|
|
||||||
self.font_path = font_path
|
self.font_path = font_path
|
||||||
self.font_dir = Path(font_dir or "resources/font")
|
self.font_dir = Path(font_dir or "resources/font")
|
||||||
self.deployment = deployment
|
self.deployment = deployment
|
||||||
self.image_format = image_format
|
self.image_format = image_format
|
||||||
self.store = SQLiteStore(self.data_dir / "farm-preview.sqlite3")
|
self.store = SQLiteStore(self.data_dir / "farm.sqlite3")
|
||||||
self.service = FarmService(self.store, Catalogue.bundled())
|
self.service = FarmService(self.store, Catalogue.bundled())
|
||||||
self.rendering = None
|
self.rendering = None
|
||||||
self.started = False
|
self.started = False
|
||||||
self.log_handler = None
|
self.log_handler = None
|
||||||
|
self.legacy_source = self.data_dir / 'farm.db'
|
||||||
|
|
||||||
async def start(self):
|
async def start(self):
|
||||||
if self.started:
|
if self.started:
|
||||||
return
|
return
|
||||||
migrated = prepare_directory(self.data_dir, self.legacy_data_dir)
|
self.data_dir.mkdir(parents=True, exist_ok=True)
|
||||||
self.log_handler = logging.FileHandler(
|
self.log_handler = logging.FileHandler(
|
||||||
self.data_dir / "farm-preview.log", encoding="utf-8",
|
self.data_dir / "farm-preview.log", encoding="utf-8",
|
||||||
)
|
)
|
||||||
@@ -94,8 +97,6 @@ class Runtime:
|
|||||||
))
|
))
|
||||||
logging.getLogger(__package__).setLevel(logging.INFO)
|
logging.getLogger(__package__).setLevel(logging.INFO)
|
||||||
logging.getLogger(__package__).addHandler(self.log_handler)
|
logging.getLogger(__package__).addHandler(self.log_handler)
|
||||||
if migrated:
|
|
||||||
logger.info('farm data directory migrated farm_preview -> farm_db')
|
|
||||||
try:
|
try:
|
||||||
await self.store.initialize(catalogue=self.service.catalogue)
|
await self.store.initialize(catalogue=self.service.catalogue)
|
||||||
except Exception:
|
except Exception:
|
||||||
@@ -148,6 +149,18 @@ class Runtime:
|
|||||||
return "农场尚未初始化,请检查启动日志。"
|
return "农场尚未初始化,请检查启动日志。"
|
||||||
try:
|
try:
|
||||||
who = Identity(self.deployment, str(bot_id), platform, str(user_id))
|
who = Identity(self.deployment, str(bot_id), platform, str(user_id))
|
||||||
|
try:
|
||||||
|
await self.store.migrate_legacy(
|
||||||
|
self.legacy_source, who, self.service.catalogue,
|
||||||
|
self.service.clock(),
|
||||||
|
)
|
||||||
|
except (FarmError, KeyError, TypeError) as error:
|
||||||
|
logger.exception('farm legacy migration failed')
|
||||||
|
code = getattr(error, 'code', '')
|
||||||
|
return ERRORS.get(
|
||||||
|
code if code.startswith('legacy_') else '',
|
||||||
|
'旧农场数据无法迁移,请联系管理员检查日志。',
|
||||||
|
)
|
||||||
args = shlex.split(arguments)
|
args = shlex.split(arguments)
|
||||||
if action == "farm" and args:
|
if action == "farm" and args:
|
||||||
aliases = {
|
aliases = {
|
||||||
|
|||||||
@@ -0,0 +1,128 @@
|
|||||||
|
"""只读旧玩家库并在调用方事务内转换单个用户。"""
|
||||||
|
|
||||||
|
from contextlib import closing
|
||||||
|
from dataclasses import asdict
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import sqlite3
|
||||||
|
|
||||||
|
from .domain import Asset, FarmError, Plot, integer, text
|
||||||
|
from .repository import SQLiteRepository
|
||||||
|
|
||||||
|
|
||||||
|
def migration_id(user_id):
|
||||||
|
return 'legacy-v1:' + hashlib.sha256(user_id.encode()).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def read_user(path, user_id):
|
||||||
|
path = Path(path).resolve()
|
||||||
|
if not path.is_file():
|
||||||
|
return None
|
||||||
|
with closing(sqlite3.connect(path.as_uri() + '?mode=ro', uri=True)) as db:
|
||||||
|
db.row_factory = sqlite3.Row
|
||||||
|
db.execute('BEGIN')
|
||||||
|
tables = {row[0] for row in db.execute(
|
||||||
|
"SELECT name FROM sqlite_master WHERE type='table'"
|
||||||
|
)}
|
||||||
|
required = {'user', 'userSeed', 'userPlant', 'userSoil'}
|
||||||
|
if not required <= tables:
|
||||||
|
raise FarmError('legacy_schema_unsupported')
|
||||||
|
users = db.execute(
|
||||||
|
'SELECT * FROM user WHERE uid=?', (user_id,),
|
||||||
|
).fetchall()
|
||||||
|
if not users:
|
||||||
|
return None
|
||||||
|
if len(users) != 1:
|
||||||
|
raise FarmError('legacy_data_invalid')
|
||||||
|
archive = {}
|
||||||
|
for table in sorted(tables):
|
||||||
|
quoted = '"' + table.replace('"', '""') + '"'
|
||||||
|
columns = {row[1] for row in db.execute(
|
||||||
|
f'PRAGMA table_info({quoted})'
|
||||||
|
)}
|
||||||
|
if 'uid' in columns:
|
||||||
|
archive[table] = [dict(row) for row in db.execute(
|
||||||
|
f'SELECT * FROM {quoted} WHERE uid=?', (user_id,),
|
||||||
|
)]
|
||||||
|
return archive
|
||||||
|
|
||||||
|
|
||||||
|
def legacy_plot(row, catalogue):
|
||||||
|
slot = integer(row['soilIndex'], 1, 10_000)
|
||||||
|
level = integer(row['soilLevel'], 0, 3)
|
||||||
|
bonuses = dict(yield_bonus=(10, 10, 20, 28)[level],
|
||||||
|
experience_bonus=28 if level == 3 else 0)
|
||||||
|
planted = row['isSoilPlanted']
|
||||||
|
if planted not in (None, 0, 1):
|
||||||
|
raise FarmError('legacy_data_invalid')
|
||||||
|
if not row['plantName']:
|
||||||
|
if planted == 1:
|
||||||
|
raise FarmError('legacy_data_invalid')
|
||||||
|
return Plot(slot, **bonuses)
|
||||||
|
plant = catalogue.get(row['plantName'])
|
||||||
|
count = integer(row['harvestCount'])
|
||||||
|
wilted = integer(row['wiltStatus'], 0, 1)
|
||||||
|
pests = integer(row['bugStatus'], 0, 1)
|
||||||
|
mature = integer(row['matureTime'])
|
||||||
|
return Plot(
|
||||||
|
slot, plant, mature, max(count, plant.crop) if wilted else count,
|
||||||
|
**bonuses, pest_count=pests, pest_spawned=pests,
|
||||||
|
maturity_pests=pests,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def import_user(db, identity, archive, catalogue, now):
|
||||||
|
operation = migration_id(identity.user_id)
|
||||||
|
if db.execute('SELECT 1 FROM accounts WHERE identity=?',
|
||||||
|
(identity.key,)).fetchone():
|
||||||
|
raise FarmError('legacy_account_conflict')
|
||||||
|
user = archive['user'][0]
|
||||||
|
text(user['name'], 64)
|
||||||
|
slots = integer(user['soil'], 1, 10_000)
|
||||||
|
balances = {
|
||||||
|
Asset.POINT: integer(user['point']),
|
||||||
|
Asset.VIP_POINT: integer(user['vipPoint']),
|
||||||
|
Asset.EXPERIENCE: integer(user['exp']),
|
||||||
|
}
|
||||||
|
db.execute('INSERT INTO accounts(identity, name) VALUES (?, ?)',
|
||||||
|
(identity.key, user['name']))
|
||||||
|
repo = SQLiteRepository(db, identity.key, operation, 'legacy_import', now)
|
||||||
|
for asset, balance in balances.items():
|
||||||
|
repo.asset(asset, balance)
|
||||||
|
inventory = set()
|
||||||
|
for table, name, quantity in (
|
||||||
|
('userSeed', 'seed', 'seeds'), ('userPlant', 'plant', 'crops'),
|
||||||
|
):
|
||||||
|
for row in archive[table]:
|
||||||
|
plant = catalogue.get(row[name])
|
||||||
|
key = (plant.plant_id, quantity)
|
||||||
|
if key in inventory:
|
||||||
|
raise FarmError('legacy_data_invalid')
|
||||||
|
inventory.add(key)
|
||||||
|
repo.stock(plant, **{quantity: integer(row['count'])})
|
||||||
|
if table == 'userPlant':
|
||||||
|
repo.lock(plant, integer(row['isLock']) != 0)
|
||||||
|
plots = {}
|
||||||
|
for row in archive['userSoil']:
|
||||||
|
plot = legacy_plot(row, catalogue)
|
||||||
|
if plot.slot in plots or plot.slot > slots:
|
||||||
|
raise FarmError('legacy_data_invalid')
|
||||||
|
plots[plot.slot] = plot
|
||||||
|
db.executemany('INSERT INTO plots(identity, slot) VALUES (?, ?)',
|
||||||
|
[(identity.key, slot) for slot in range(1, slots + 1)])
|
||||||
|
for plot in plots.values():
|
||||||
|
repo.save_plot(plot)
|
||||||
|
result = {
|
||||||
|
'migration_version': 1, 'account': asdict(repo.account()),
|
||||||
|
'legacy_rows': archive,
|
||||||
|
'unsupported_tables': sorted(set(archive) - {
|
||||||
|
'user', 'userSeed', 'userPlant', 'userSoil',
|
||||||
|
}),
|
||||||
|
}
|
||||||
|
db.execute(
|
||||||
|
'INSERT INTO operations VALUES (?, ?, ?, ?, ?, ?)',
|
||||||
|
(identity.key, operation, 'legacy_import', operation,
|
||||||
|
json.dumps(result, ensure_ascii=False), now),
|
||||||
|
)
|
||||||
|
return True
|
||||||
@@ -252,6 +252,48 @@ class SQLiteStore:
|
|||||||
if not self._ready:
|
if not self._ready:
|
||||||
raise StorageError("store_not_initialized_or_closed")
|
raise StorageError("store_not_initialized_or_closed")
|
||||||
|
|
||||||
|
async def migrate_legacy(self, source, identity, catalogue, now):
|
||||||
|
from .legacy import import_user, migration_id, read_user
|
||||||
|
|
||||||
|
self._ensure_ready()
|
||||||
|
|
||||||
|
def execute():
|
||||||
|
with self._connection() as connection:
|
||||||
|
self._validate(connection)
|
||||||
|
connection.execute('BEGIN IMMEDIATE')
|
||||||
|
try:
|
||||||
|
bound = connection.execute(
|
||||||
|
"SELECT identity FROM operations "
|
||||||
|
"WHERE operation_id=? AND action='legacy_import'",
|
||||||
|
(migration_id(identity.user_id),),
|
||||||
|
).fetchall()
|
||||||
|
if bound:
|
||||||
|
if any(row[0] != identity.key for row in bound):
|
||||||
|
raise FarmError('legacy_identity_bound')
|
||||||
|
connection.commit()
|
||||||
|
return False
|
||||||
|
archive = read_user(source, identity.user_id)
|
||||||
|
migrated = bool(archive) and import_user(
|
||||||
|
connection, identity, archive, catalogue, now,
|
||||||
|
)
|
||||||
|
connection.commit()
|
||||||
|
if migrated:
|
||||||
|
LOGGER.info(
|
||||||
|
'farm legacy migration committed operation=%s '
|
||||||
|
'tables=%s', migration_id(identity.user_id),
|
||||||
|
{name: len(rows) for name, rows in archive.items()},
|
||||||
|
)
|
||||||
|
return migrated
|
||||||
|
except BaseException:
|
||||||
|
connection.rollback()
|
||||||
|
LOGGER.exception(
|
||||||
|
'farm legacy migration rolled back operation=%s',
|
||||||
|
migration_id(identity.user_id),
|
||||||
|
)
|
||||||
|
raise
|
||||||
|
|
||||||
|
return await self._offload(execute)
|
||||||
|
|
||||||
async def transact(
|
async def transact(
|
||||||
self, identity: Identity, operation_id: str, action: str,
|
self, identity: Identity, operation_id: str, action: str,
|
||||||
parameters: dict, now: int, operation,
|
parameters: dict, now: int, operation,
|
||||||
|
|||||||
@@ -1,5 +1,4 @@
|
|||||||
import importlib.util
|
import importlib.util
|
||||||
import os
|
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
import tempfile
|
import tempfile
|
||||||
import unittest
|
import unittest
|
||||||
@@ -20,66 +19,8 @@ class DataDirectoryTests(unittest.TestCase):
|
|||||||
self.base = Path(self.temp.name)
|
self.base = Path(self.temp.name)
|
||||||
|
|
||||||
def test_default_and_explicit_directory(self):
|
def test_default_and_explicit_directory(self):
|
||||||
self.assertEqual(self.module.data_directories(self.base), (
|
self.assertEqual(
|
||||||
self.base / 'farm_db', self.base / 'farm_preview',
|
self.module.data_directory(self.base), self.base / 'farm_db',
|
||||||
))
|
|
||||||
custom = self.base / 'custom'
|
|
||||||
self.assertEqual(self.module.data_directories(self.base, custom), (
|
|
||||||
custom, None,
|
|
||||||
))
|
|
||||||
|
|
||||||
def test_migrate_preserves_database_log_and_sidecars(self):
|
|
||||||
old = self.base / 'farm_preview'
|
|
||||||
old.mkdir()
|
|
||||||
for name in ('farm-preview.sqlite3', 'farm-preview.sqlite3-wal',
|
|
||||||
'farm-preview.log'):
|
|
||||||
(old / name).write_bytes(name.encode())
|
|
||||||
target = self.base / 'farm_db'
|
|
||||||
self.assertTrue(self.module.prepare_directory(target, old))
|
|
||||||
self.assertFalse(old.exists())
|
|
||||||
for path in target.iterdir():
|
|
||||||
self.assertEqual(path.read_bytes(), path.name.encode())
|
|
||||||
self.assertFalse(self.module.prepare_directory(target, old))
|
|
||||||
|
|
||||||
def test_conflicting_directory_stops_without_overwrite(self):
|
|
||||||
old, target = self.base / 'farm_preview', self.base / 'farm_db'
|
|
||||||
old.mkdir()
|
|
||||||
target.mkdir()
|
|
||||||
(old / 'farm-preview.sqlite3').write_bytes(b'old')
|
|
||||||
(target / 'farm-preview.sqlite3').write_bytes(b'new')
|
|
||||||
with self.assertRaises(FileExistsError):
|
|
||||||
self.module.prepare_directory(target, old)
|
|
||||||
self.assertEqual((old / 'farm-preview.sqlite3').read_bytes(), b'old')
|
|
||||||
self.assertEqual((target / 'farm-preview.sqlite3').read_bytes(), b'new')
|
|
||||||
|
|
||||||
def test_migration_rejects_paths_outside_default_siblings(self):
|
|
||||||
with self.assertRaises(ValueError):
|
|
||||||
self.module.prepare_directory(
|
|
||||||
self.base / 'farm_db', self.base / 'other/farm_preview',
|
|
||||||
)
|
|
||||||
|
|
||||||
@unittest.skipUnless(os.name == 'nt', 'Windows short paths only')
|
|
||||||
def test_migration_accepts_short_and_long_parent_paths(self):
|
|
||||||
import ctypes
|
|
||||||
|
|
||||||
parent = self.base / 'Farm migration long directory'
|
|
||||||
parent.mkdir()
|
|
||||||
buffer = ctypes.create_unicode_buffer(32768)
|
|
||||||
get_short_path = ctypes.windll.kernel32.GetShortPathNameW
|
|
||||||
get_short_path.argtypes = (
|
|
||||||
ctypes.c_wchar_p, ctypes.c_wchar_p, ctypes.c_uint,
|
|
||||||
)
|
)
|
||||||
get_short_path.restype = ctypes.c_uint
|
custom = self.base / 'custom'
|
||||||
length = get_short_path(str(parent), buffer, len(buffer))
|
self.assertEqual(self.module.data_directory(self.base, custom), custom)
|
||||||
self.assertTrue(0 < length < len(buffer))
|
|
||||||
short_parent = Path(buffer.value)
|
|
||||||
if short_parent == parent.resolve():
|
|
||||||
self.skipTest('Short path aliases are disabled on this volume')
|
|
||||||
old = short_parent / 'farm_preview'
|
|
||||||
old.mkdir()
|
|
||||||
(old / 'farm-preview.sqlite3').write_bytes(b'preserved')
|
|
||||||
target = parent.resolve() / 'farm_db'
|
|
||||||
self.assertTrue(self.module.prepare_directory(target, old))
|
|
||||||
self.assertEqual((target / 'farm-preview.sqlite3').read_bytes(),
|
|
||||||
b'preserved')
|
|
||||||
self.assertFalse(old.exists())
|
|
||||||
|
|||||||
@@ -0,0 +1,180 @@
|
|||||||
|
import asyncio
|
||||||
|
from contextlib import closing
|
||||||
|
from dataclasses import replace
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import sqlite3
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
sys.path.insert(0, str(ROOT / 'src'))
|
||||||
|
|
||||||
|
from farm_core import Catalogue, FarmService, Identity, SQLiteStore
|
||||||
|
from farm_core.domain import FarmError
|
||||||
|
|
||||||
|
|
||||||
|
def create_legacy(path):
|
||||||
|
with closing(sqlite3.connect(path)) as db, db:
|
||||||
|
db.executescript('''
|
||||||
|
CREATE TABLE user(uid TEXT PRIMARY KEY, name TEXT, exp INTEGER,
|
||||||
|
point INTEGER, vipPoint INTEGER, soil INTEGER);
|
||||||
|
CREATE TABLE userSeed(uid TEXT, seed TEXT, count INTEGER);
|
||||||
|
CREATE TABLE userPlant(uid TEXT, plant TEXT, count INTEGER,
|
||||||
|
isLock INTEGER);
|
||||||
|
CREATE TABLE userSoil(uid TEXT, soilIndex INTEGER, plantName TEXT,
|
||||||
|
matureTime INTEGER, soilLevel INTEGER, wiltStatus INTEGER,
|
||||||
|
bugStatus INTEGER, harvestCount INTEGER, isSoilPlanted INTEGER);
|
||||||
|
CREATE TABLE userItem(uid TEXT, item TEXT, count INTEGER);
|
||||||
|
INSERT INTO user VALUES ('1', '旧农场', 900, 1234, 56, 4);
|
||||||
|
INSERT INTO userSeed VALUES ('1', '白萝卜', 12);
|
||||||
|
INSERT INTO userPlant VALUES ('1', '白萝卜', 23, 1);
|
||||||
|
INSERT INTO userSoil VALUES
|
||||||
|
('1', 1, '白萝卜', 2000, 2, 0, 1, 0, 1),
|
||||||
|
('1', 2, '白萝卜', 1000, 3, 1, 0, 0, 1),
|
||||||
|
('1', 3, '', 0, 0, 0, 0, 0, 0);
|
||||||
|
INSERT INTO userItem VALUES ('1', '化肥', 7);
|
||||||
|
''')
|
||||||
|
|
||||||
|
|
||||||
|
class LegacyMigrationTests(unittest.IsolatedAsyncioTestCase):
|
||||||
|
async def asyncSetUp(self):
|
||||||
|
self.temp = tempfile.TemporaryDirectory()
|
||||||
|
self.root = Path(self.temp.name)
|
||||||
|
self.source = self.root / 'farm.db'
|
||||||
|
create_legacy(self.source)
|
||||||
|
self.catalogue = Catalogue.bundled()
|
||||||
|
self.store = SQLiteStore(self.root / 'farm.sqlite3')
|
||||||
|
await self.store.initialize(self.catalogue)
|
||||||
|
self.who = Identity('zhenxun-preview', 'bot', 'OneBot V11', '1')
|
||||||
|
|
||||||
|
async def asyncTearDown(self):
|
||||||
|
await self.store.close()
|
||||||
|
self.temp.cleanup()
|
||||||
|
|
||||||
|
async def migrate(self, who=None):
|
||||||
|
return await self.store.migrate_legacy(
|
||||||
|
self.source, who or self.who, self.catalogue, 1500,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def assert_empty(self):
|
||||||
|
with closing(sqlite3.connect(self.store.path)) as db, db:
|
||||||
|
for table in ('accounts', 'inventory', 'plots', 'operations',
|
||||||
|
'ledger'):
|
||||||
|
self.assertEqual(db.execute(
|
||||||
|
f'SELECT COUNT(*) FROM {table}'
|
||||||
|
).fetchone()[0], 0)
|
||||||
|
|
||||||
|
async def test_assets_plots_archive_and_source_preserved(self):
|
||||||
|
before = self.source.read_bytes()
|
||||||
|
self.assertTrue(await self.migrate())
|
||||||
|
snapshot = await self.store.snapshot(self.who)
|
||||||
|
self.assertEqual((snapshot.account.name, snapshot.account.point,
|
||||||
|
snapshot.account.vip_point,
|
||||||
|
snapshot.account.experience),
|
||||||
|
('旧农场', 1234, 56, 900))
|
||||||
|
item = snapshot.inventory[0]
|
||||||
|
self.assertEqual((item.seeds, item.crops, item.locked), (12, 23, True))
|
||||||
|
self.assertEqual(len(snapshot.plots), 4)
|
||||||
|
self.assertEqual(snapshot.plots[0].mature_at, 2000)
|
||||||
|
self.assertEqual(snapshot.plots[0].pest_count, 1)
|
||||||
|
self.assertEqual(snapshot.plots[0].yield_bonus, 20)
|
||||||
|
self.assertTrue(snapshot.plots[1].withered)
|
||||||
|
self.assertEqual(snapshot.plots[1].experience_bonus, 28)
|
||||||
|
self.assertIsNone(snapshot.plots[3].plant)
|
||||||
|
with closing(sqlite3.connect(self.store.path)) as db, db:
|
||||||
|
result = json.loads(db.execute(
|
||||||
|
"SELECT result FROM operations WHERE action='legacy_import'"
|
||||||
|
).fetchone()[0])
|
||||||
|
self.assertEqual(result['legacy_rows']['userItem'][0]['count'], 7)
|
||||||
|
self.assertEqual(result['unsupported_tables'], ['userItem'])
|
||||||
|
self.assertEqual(db.execute(
|
||||||
|
'SELECT SUM(delta) FROM ledger WHERE asset=?', ('point',),
|
||||||
|
).fetchone()[0], 1234)
|
||||||
|
self.assertEqual(db.execute('PRAGMA foreign_key_check').fetchall(),
|
||||||
|
[])
|
||||||
|
self.assertEqual(self.source.read_bytes(), before)
|
||||||
|
|
||||||
|
async def test_repeated_import_does_not_restore_spent_assets(self):
|
||||||
|
await self.migrate()
|
||||||
|
service = FarmService(self.store, self.catalogue, clock=lambda: 1500)
|
||||||
|
await service.buy(self.who, '白萝卜', 1, 'buy')
|
||||||
|
before = await self.store.snapshot(self.who)
|
||||||
|
self.assertFalse(await self.migrate())
|
||||||
|
self.assertEqual(await self.store.snapshot(self.who), before)
|
||||||
|
await self.store.close()
|
||||||
|
self.store = SQLiteStore(self.root / 'farm.sqlite3')
|
||||||
|
await self.store.initialize(self.catalogue)
|
||||||
|
self.source.rename(self.root / 'farm.db.saved')
|
||||||
|
self.assertFalse(await self.migrate())
|
||||||
|
self.assertEqual(await self.store.snapshot(self.who), before)
|
||||||
|
|
||||||
|
async def test_concurrent_identities_bind_only_once(self):
|
||||||
|
other = Identity('zhenxun-preview', 'other', 'QQ', '1')
|
||||||
|
results = await asyncio.gather(
|
||||||
|
self.migrate(), self.migrate(other), return_exceptions=True,
|
||||||
|
)
|
||||||
|
self.assertEqual(sum(result is True for result in results), 1)
|
||||||
|
failures = [r for r in results if isinstance(r, FarmError)]
|
||||||
|
self.assertEqual([r.code for r in failures], ['legacy_identity_bound'])
|
||||||
|
with closing(sqlite3.connect(self.store.path)) as db, db:
|
||||||
|
self.assertEqual(db.execute(
|
||||||
|
'SELECT COUNT(*) FROM accounts'
|
||||||
|
).fetchone()[0], 1)
|
||||||
|
|
||||||
|
async def test_existing_account_is_not_overwritten(self):
|
||||||
|
service = FarmService(self.store, self.catalogue, clock=lambda: 1500)
|
||||||
|
await service.register(self.who, '新农场', 'new')
|
||||||
|
before = await self.store.snapshot(self.who)
|
||||||
|
with self.assertRaisesRegex(FarmError, 'legacy_account_conflict'):
|
||||||
|
await self.migrate()
|
||||||
|
self.assertEqual(await self.store.snapshot(self.who), before)
|
||||||
|
|
||||||
|
async def test_unknown_crop_rolls_back_all_tables(self):
|
||||||
|
with closing(sqlite3.connect(self.source)) as db, db:
|
||||||
|
db.execute("UPDATE userSeed SET seed='不存在的作物'")
|
||||||
|
with self.assertRaisesRegex(FarmError, 'unknown_plant'):
|
||||||
|
await self.migrate()
|
||||||
|
await self.assert_empty()
|
||||||
|
|
||||||
|
async def test_negative_balance_is_rejected(self):
|
||||||
|
with closing(sqlite3.connect(self.source)) as db, db:
|
||||||
|
db.execute('UPDATE user SET point=-1')
|
||||||
|
with self.assertRaisesRegex(FarmError, 'invalid_integer'):
|
||||||
|
await self.migrate()
|
||||||
|
await self.assert_empty()
|
||||||
|
|
||||||
|
async def test_ambiguous_crop_is_not_guessed(self):
|
||||||
|
plant = self.catalogue.get('白萝卜')
|
||||||
|
self.catalogue = Catalogue((plant, replace(plant, plant_id=999999)))
|
||||||
|
with self.assertRaisesRegex(FarmError, 'ambiguous_plant'):
|
||||||
|
await self.migrate()
|
||||||
|
await self.assert_empty()
|
||||||
|
|
||||||
|
async def test_duplicate_plot_rolls_back_and_can_retry_after_repair(self):
|
||||||
|
with closing(sqlite3.connect(self.source)) as db, db:
|
||||||
|
db.execute('INSERT INTO userSoil SELECT * FROM userSoil '
|
||||||
|
'WHERE soilIndex=1')
|
||||||
|
with self.assertRaisesRegex(FarmError, 'legacy_data_invalid'):
|
||||||
|
await self.migrate()
|
||||||
|
await self.assert_empty()
|
||||||
|
with closing(sqlite3.connect(self.source)) as db, db:
|
||||||
|
db.execute('DELETE FROM userSoil WHERE rowid='
|
||||||
|
'(SELECT MAX(rowid) FROM userSoil)')
|
||||||
|
self.assertTrue(await self.migrate())
|
||||||
|
|
||||||
|
async def test_missing_source_or_user_does_not_create_account(self):
|
||||||
|
self.assertFalse(await self.migrate(
|
||||||
|
Identity('zhenxun-preview', 'bot', 'OneBot V11', 'missing'),
|
||||||
|
))
|
||||||
|
self.source.unlink()
|
||||||
|
self.assertFalse(await self.migrate())
|
||||||
|
await self.assert_empty()
|
||||||
|
|
||||||
|
async def test_missing_schema_is_rejected(self):
|
||||||
|
with closing(sqlite3.connect(self.source)) as db, db:
|
||||||
|
db.execute('DROP TABLE userSeed')
|
||||||
|
with self.assertRaisesRegex(FarmError, 'legacy_schema_unsupported'):
|
||||||
|
await self.migrate()
|
||||||
|
await self.assert_empty()
|
||||||
@@ -1,4 +1,5 @@
|
|||||||
import asyncio
|
import asyncio
|
||||||
|
from contextlib import closing
|
||||||
from dataclasses import replace
|
from dataclasses import replace
|
||||||
import importlib
|
import importlib
|
||||||
import importlib.util
|
import importlib.util
|
||||||
@@ -216,29 +217,61 @@ class ZhenxunRuntimeTests(unittest.IsolatedAsyncioTestCase):
|
|||||||
)
|
)
|
||||||
self.assertEqual(before, await self.call("inventory"))
|
self.assertEqual(before, await self.call("inventory"))
|
||||||
|
|
||||||
async def test_default_directory_migration_keeps_account_and_inventory(self):
|
async def test_existing_preview_directory_is_ignored(self):
|
||||||
old = self.root / 'migration/farm_preview'
|
old = self.root / 'directories/farm_preview'
|
||||||
target = old.with_name('farm_db')
|
target = old.with_name('farm_db')
|
||||||
previous = self.module.Runtime(old, choose_font())
|
previous = self.module.Runtime(old, choose_font())
|
||||||
await previous.start()
|
await previous.start()
|
||||||
try:
|
try:
|
||||||
await previous.execute('register', '迁移农场', **self.context)
|
await previous.execute('register', '旧农场', **self.context)
|
||||||
before = await previous.execute('inventory', '', **self.context)
|
|
||||||
finally:
|
finally:
|
||||||
await previous.close()
|
await previous.close()
|
||||||
current = self.module.Runtime(
|
old_database = old / 'farm.sqlite3'
|
||||||
target, choose_font(), legacy_data_dir=old,
|
before = old_database.read_bytes()
|
||||||
)
|
target.mkdir()
|
||||||
|
current = self.module.Runtime(target, choose_font())
|
||||||
await current.start()
|
await current.start()
|
||||||
try:
|
try:
|
||||||
self.assertFalse(old.exists())
|
self.assertEqual(old_database.read_bytes(), before)
|
||||||
self.assertTrue((target / 'farm-preview.sqlite3').is_file())
|
self.assertTrue((target / 'farm.sqlite3').is_file())
|
||||||
self.assertEqual(await current.execute(
|
self.assertEqual(await current.execute(
|
||||||
'inventory', '', **self.context,
|
'inventory', '', **self.context,
|
||||||
), before)
|
), '请先发送:开通农场')
|
||||||
finally:
|
finally:
|
||||||
await current.close()
|
await current.close()
|
||||||
|
|
||||||
|
async def test_first_command_imports_legacy_and_binds_identity(self):
|
||||||
|
from test_legacy_migration import create_legacy
|
||||||
|
|
||||||
|
source = self.root / 'data/farm.db'
|
||||||
|
create_legacy(source)
|
||||||
|
before = source.read_bytes()
|
||||||
|
result = await self.call('inventory')
|
||||||
|
self.assertIn('旧农场', result)
|
||||||
|
self.assertIn('1234', result)
|
||||||
|
self.assertTrue((self.root / 'data/farm.sqlite3').is_file())
|
||||||
|
self.assertFalse((self.root / 'data/farm-preview.sqlite3').exists())
|
||||||
|
self.assertEqual(await self.call('inventory'), result)
|
||||||
|
self.assertIn('已绑定其他机器人或平台', await self.call(
|
||||||
|
'register', bot_id='other',
|
||||||
|
))
|
||||||
|
self.assertEqual(source.read_bytes(), before)
|
||||||
|
|
||||||
|
async def test_failed_migration_does_not_register_empty_account(self):
|
||||||
|
from test_legacy_migration import create_legacy
|
||||||
|
import sqlite3
|
||||||
|
|
||||||
|
source = self.root / 'data/farm.db'
|
||||||
|
create_legacy(source)
|
||||||
|
with closing(sqlite3.connect(source)) as db, db:
|
||||||
|
db.execute("UPDATE userSeed SET seed='不存在的作物'")
|
||||||
|
self.assertIn('无法迁移', await self.call('register'))
|
||||||
|
identity = self.module.Identity(
|
||||||
|
'zhenxun-preview', 'bot', 'OneBot V11', '1',
|
||||||
|
)
|
||||||
|
with self.assertRaisesRegex(self.module.FarmError, 'not_registered'):
|
||||||
|
await self.runtime.store.snapshot(identity)
|
||||||
|
|
||||||
async def test_mock_host_load_registers_hooks_and_original_matchers(self):
|
async def test_mock_host_load_registers_hooks_and_original_matchers(self):
|
||||||
hooks = {"start":[], "close":[], "commands":{}}
|
hooks = {"start":[], "close":[], "commands":{}}
|
||||||
driver = SimpleNamespace(
|
driver = SimpleNamespace(
|
||||||
|
|||||||
Reference in New Issue
Block a user