From dffb20e13527c1e0f58949e1d5f39cffa6191b16 Mon Sep 17 00:00:00 2001 From: "H.W." Date: Fri, 14 Aug 2026 03:40:48 -0700 Subject: [PATCH 01/13] =?UTF-8?q?ci:=20=E7=A6=81=E6=AD=A2=E6=96=B0?= =?UTF-8?q?=E5=A2=9E=E9=9D=9E=20ASCII=20=E8=B5=84=E6=BA=90=E6=96=87?= =?UTF-8?q?=E4=BB=B6=E5=90=8D=EF=BC=88macOS=20=E7=AD=BE=E5=90=8D=E6=A3=98?= =?UTF-8?q?=E8=BD=AE=EF=BC=89?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Nuitka --mode=app 把整个 payload 放在 Contents/MacOS/ 下，codesign 的 默认规则把该目录里的每个文件都当作 nested code 逐个签名，并往 CodeResources 里写一条 `identifier ...` 需求。文件名一旦是非 ASCII，codesign 会把 identifier 写成十六进制字面量，那不是合法的 requirement 语法，整个 bundle 随后校验为 "the sealed resource directory is invalid"，签名、公证、Steam 上传全部卡死，而报错里不带任何路径。 CI 结构上抓不到这个问题：build-desktop.yml 用 ad-hoc 签名（--sign -）， 其 requirement 是 cdhash H"..." 不含 identifier，所以永远是绿的，只有 本地 Developer ID 路径（build_mac.sh）才会踩。这条 lint 就是补上 CI 产生不了的信号。 棘轮而非禁令：存量资源记在 scripts/nonascii_asset_baseline.txt，该清单 只许缩短。扫描范围只覆盖真正随包分发的内容——git 已知的 bundled roots 下的文件（含未 add 的新文件，所以 git add 之前就会报），以及构建时会解包 进这些目录的压缩包成员名，读成员名无需真正解包，保证新检出和构建机上 结论一致。 Co-Authored-By: Claude Opus 5 --- .github/workflows/analyze.yml | 18 + scripts/check_no_nonascii_asset_names.py | 437 ++++++++++++++++++ scripts/nonascii_asset_baseline.txt | 352 ++++++++++++++ .../test_check_no_nonascii_asset_names.py | 274 +++++++++++ 4 files changed, 1081 insertions(+) create mode 100644 scripts/check_no_nonascii_asset_names.py create mode 100644 scripts/nonascii_asset_baseline.txt create mode 100644 tests/unit/test_check_no_nonascii_asset_names.py diff --git a/.github/workflows/analyze.yml b/.github/workflows/analyze.yml index 9f0f64326b..50cfce090c 100644 --- a/.github/workflows/analyze.yml +++ b/.github/workflows/analyze.yml @@ -278,6 +278,24 @@ jobs: # link, or move the referenced content into docs/. run: python scripts/check_docs_no_relative_paths.py + - name: Forbid new non-ASCII filenames in bundled assets + # Nuitka --mode=app puts the payload under Contents/MacOS/, where + # codesign treats every file as nested code and writes an + # `identifier ...` requirement into CodeResources. A non-ASCII + # name becomes a hex literal, which is not valid requirement syntax, + # and the whole bundle then fails to verify with "the sealed resource + # directory is invalid" — dead in the water for signing, notarization + # and Steam upload, with no filename in the error to go on. + # + # This step exists because build-desktop.yml cannot catch it: it signs + # ad-hoc (`--sign -`), whose requirements are `cdhash H"..."` and carry + # no identifier, so the bug is invisible there and only bites the local + # Developer ID path (build_mac.sh). Ratchet, not a ban — the assets + # that predate the check live in scripts/nonascii_asset_baseline.txt + # and that list may only shrink. Companion unit test: + # tests/unit/test_check_no_nonascii_asset_names.py. + run: python scripts/check_no_nonascii_asset_names.py + core-contracts: name: Core package contracts runs-on: ubuntu-latest diff --git a/scripts/check_no_nonascii_asset_names.py b/scripts/check_no_nonascii_asset_names.py new file mode 100644 index 0000000000..50a7e540e8 --- /dev/null +++ b/scripts/check_no_nonascii_asset_names.py @@ -0,0 +1,437 @@ +#!/usr/bin/env python3 +# Copyright 2025-2026 Project N.E.K.O. Team +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Ratchet: no NEW non-ASCII filenames in anything bundled into the desktop app. + +Why this exists +--------------- +Nuitka ``--mode=app`` puts the whole payload under +``projectneko_server.app/Contents/MacOS/``. codesign's default bundle rules +(``rules2``) classify everything under that directory as *nested code*:: + + ^(Frameworks|SharedFrameworks|PlugIns|Plug-ins|XPCServices|Helpers|MacOS| + Library/(Automator|Spotlight|LoginItems))/ => nested + +So every .mp3 / .png / .json we ship is signed as its own code object, and for +each one codesign writes a designated requirement into +``Contents/_CodeSignature/CodeResources``:: + + identifier and anchor apple generic and certificate ... + +When the filename is non-ASCII, codesign emits that identifier as a **hex +literal** instead of a quoted string:: + + identifier 0xe4b883e5a4a9e5898defbc8ce68891e4bbace8bf98e58faae698afe7a... + +That is not valid requirement syntax. Reading the seal back fails, and the +whole bundle then verifies as:: + + : the sealed resource directory is invalid + +…which blocks signing, notarization, and Steam upload. One such file poisons +the entire bundle; the error names no path, so it is genuinely painful to +diagnose from scratch. + +Why CI can never catch it on its own +------------------------------------ +``.github/workflows/build-desktop.yml`` signs ad-hoc:: + + codesign --force --deep --sign - dist/Xiao8/projectneko_server.app + +Ad-hoc signatures use ``cdhash H"..."`` requirements, which carry no +identifier at all — so the hex-literal bug simply does not occur there and +the workflow is always green. The breakage only appears on the local +Developer ID signing path (``build_mac.sh``). This lint is the substitute for +a signal CI structurally cannot produce. + +What it scans +------------- +Only what actually ships: + +- files git knows about under the bundled roots (``BUNDLED_ROOTS``) — + tracked plus untracked-but-not-ignored, so the check fires before + ``git add`` rather than only after commit; +- member names inside the archives that get unpacked into those roots at + build time (``assets/*.tar.gz`` -> ``static//`` via + ``build_frontend.sh``; ``frontend/pngtuber-packs/*.zip`` -> + ``static/pngtuber//`` via ``scripts/unpack_builtin_pngtuber.py``). + Reading member names needs no build and no extraction, so the check gives + the same answer in a fresh checkout as on a build machine. + +Asking git (rather than walking the working tree) keeps the result +deterministic: gitignored build outputs, downloaded model weights, and local +runtime files never make the answer wander between a fresh checkout and a +built one. Pass ``--include-untracked`` to additionally walk the on-disk tree +— useful right after a build, when you want to sanity-check generated payload +too. + +Only **basenames** are checked, not whole paths. codesign derives the nested +identifier from the filename alone, so a non-ASCII *directory* holding +ASCII-named files signs and verifies cleanly (checked against a real +Developer ID certificate: ``MacOS//plain_name.png`` seals as +``identifier "plain_name"`` and passes ``--verify --deep --strict``). +Flagging those too would only produce failures nobody can act on. + +``tests/`` is deliberately out of scope: it carries a few CJK fixture paths +and never reaches the .app. + +Baseline +-------- +338 tracked files plus 1 archive member are already non-ASCII when this check +was written, across three unrelated subsystems: + +- ``static/assets/tutorial/guide-audio/{zh,ja,ko,ru,en}/*.mp3`` — filenames + are truncated line transcripts, referenced from the ``audioFilesByKey`` + manifests in ``static/tutorial/yui-guide/days/*.js``; +- ``static/vrm/motion/**``, ``static/vrm/animation/*``, + ``static/mmd/animation/*`` — ``static/vrm/motion/manifest.json`` states the + convention outright (``"fileNaming": "descriptive Chinese filename with + stable id"``, ``"authoritativeLanguage": "zh-CN"``), because motion lookup + runs through Chinese action-card retrieval; +- ``static/game/games/soccer/audio/*.mp3``, plus one Live2D expression file + under ``static/yui-origin/expressions/`` that ships inside + ``assets/yui-origin.tar.gz`` and is named from the ``.model3.json``. + +Renaming those is a product decision, not a mechanical sweep — the VRM naming +scheme in particular is load-bearing for motion retrieval. So this is a +ratchet, not a clean-room ban: everything listed in +``scripts/nonascii_asset_baseline.txt`` is grandfathered, anything new fails. + +TODO: shrink the baseline. Each family needs its own follow-up — rename the +files to ASCII (stable slug or id) and update the manifest that names them. +An empty baseline is the goal; when it gets there, delete the file and make +this a plain ban. + +Usage +----- + python scripts/check_no_nonascii_asset_names.py + python scripts/check_no_nonascii_asset_names.py --list + python scripts/check_no_nonascii_asset_names.py --include-untracked + python scripts/check_no_nonascii_asset_names.py --update-baseline +""" +from __future__ import annotations + +import argparse +import os +import subprocess +import sys +import tarfile +import zipfile +from pathlib import Path, PurePosixPath + +REPO_ROOT = Path(__file__).resolve().parent.parent +BASELINE_PATH = REPO_ROOT / "scripts" / "nonascii_asset_baseline.txt" + +CODE = "NONASCII_ASSET_NAME" + +# Directories whose contents end up inside the .app payload. Mirrors the +# --include-data-dir / --include-package-data set in build_nuitka.bat, +# build_mac.sh and .github/workflows/build-desktop.yml; keep in sync when a +# new payload directory is added there. +BUNDLED_ROOTS: tuple[str, ...] = ( + "static", + "config", + "templates", + "assets", + "data", + "docs", + "frontend", + "plugin/plugins", +) + +# Archives that are expanded into a bundled root at build time. Value is the +# bundled path prefix the members land under, so a violation reports where the +# file will actually sit inside the .app rather than where it hides today. +TAR_ARCHIVE_DESTS: tuple[tuple[str, str], ...] = ( + # build_frontend.sh: unpack_live2d_model -> static// + ("assets", "static"), +) +ZIP_ARCHIVE_DESTS: tuple[tuple[str, str], ...] = ( + # scripts/unpack_builtin_pngtuber.py -> static/pngtuber// + ("frontend/pngtuber-packs", "static/pngtuber"), +) + +# Never walked under --include-untracked. Vendored/derived trees whose names +# we do not author; a hit in here is a bug in the upstream package, not in +# this repo, and the .app build has its own gates for those. +WALK_EXCLUDE_DIRS = frozenset( + { + "node_modules", + "__pycache__", + ".git", + ".venv", + "venv", + ".mypy_cache", + ".ruff_cache", + ".pytest_cache", + } +) + + +def _is_ascii(text: str) -> bool: + return all(ord(ch) < 128 for ch in text) + + +def _under_bundled_root(rel_posix: str) -> bool: + return any( + rel_posix == root or rel_posix.startswith(root + "/") for root in BUNDLED_ROOTS + ) + + +def _git_listed_offenders(repo_root: Path) -> set[str]: + """Non-ASCII git-visible paths under the bundled roots. + + ``--cached --others --exclude-standard`` = tracked files plus untracked + ones that are not gitignored. The ``--others`` half is what makes the + check fire before ``git add``, while ``--exclude-standard`` keeps build + outputs and downloaded weights out so the answer does not depend on + whether the tree has been built. + + ``-z`` matters: without it git quote-escapes non-ASCII paths + (``"static/\\344\\270\\203....mp3"``), which is exactly the set we are + looking for. + """ + try: + raw = subprocess.run( + ["git", "ls-files", "-z", "--cached", "--others", "--exclude-standard"], + cwd=repo_root, + capture_output=True, + check=True, + ).stdout.decode("utf-8", errors="surrogateescape") + except (OSError, subprocess.CalledProcessError) as exc: + print(f"error: cannot list git files: {exc}", file=sys.stderr) + raise SystemExit(2) from exc + + return { + path + for path in raw.split("\0") + if path + and _under_bundled_root(path) + and not _is_ascii(PurePosixPath(path).name) + } + + +def _archive_offenders(repo_root: Path) -> dict[str, str]: + """Non-ASCII member names inside archives that get unpacked into the app. + + Keyed by the path the member will occupy once unpacked, so both the + baseline and the violation message point at the bundled location; the + value is the archive to edit. + """ + offenders: dict[str, str] = {} + + def _record(members: list[str], dest_prefix: str, archive_rel: str) -> None: + for member in members: + if _is_ascii(PurePosixPath(member).name): + continue + offenders[f"{dest_prefix}/{member}"] = archive_rel + + for source_dir, dest_prefix in TAR_ARCHIVE_DESTS: + directory = repo_root / source_dir + if not directory.is_dir(): + continue + for archive in sorted(directory.glob("*.tar.gz")): + with tarfile.open(archive) as handle: + names = [m.name for m in handle.getmembers() if m.isfile()] + _record(names, dest_prefix, archive.relative_to(repo_root).as_posix()) + + for source_dir, dest_prefix in ZIP_ARCHIVE_DESTS: + directory = repo_root / source_dir + if not directory.is_dir(): + continue + for archive in sorted(directory.glob("*.zip")): + with zipfile.ZipFile(archive) as handle: + names = [i.filename for i in handle.infolist() if not i.is_dir()] + _record(names, dest_prefix, archive.relative_to(repo_root).as_posix()) + + return offenders + + +def _untracked_offenders(repo_root: Path) -> set[str]: + """Non-ASCII paths found by walking the on-disk bundled roots. + + Only used with ``--include-untracked``: after a build these roots also + hold generated payload (unpacked models, vite bundles, downloaded model + weights) that no git listing can see. + """ + offenders: set[str] = set() + for root in BUNDLED_ROOTS: + base = repo_root / root + if not base.is_dir(): + continue + for current, dirs, files in os.walk(base): + dirs[:] = [d for d in dirs if d not in WALK_EXCLUDE_DIRS] + current_path = Path(current) + for name in files: + if _is_ascii(name): + continue + offenders.add( + (current_path / name).relative_to(repo_root).as_posix() + ) + return offenders + + +def collect_offenders( + repo_root: Path, include_untracked: bool = False +) -> tuple[set[str], dict[str, str]]: + """Return (bundled paths with non-ASCII names, path -> owning archive).""" + from_archives = _archive_offenders(repo_root) + offenders = _git_listed_offenders(repo_root) | set(from_archives) + if include_untracked: + offenders |= _untracked_offenders(repo_root) + return offenders, from_archives + + +def load_baseline(path: Path) -> set[str]: + if not path.is_file(): + return set() + entries: set[str] = set() + for line in path.read_text(encoding="utf-8").splitlines(): + stripped = line.strip() + if not stripped or stripped.startswith("#"): + continue + entries.add(stripped) + return entries + + +def write_baseline(path: Path, offenders: set[str]) -> None: + header = ( + "# Grandfathered non-ASCII filenames in bundled assets.\n" + "#\n" + "# See scripts/check_no_nonascii_asset_names.py for why these break macOS\n" + "# Developer ID signing (codesign emits a hex-literal `identifier` in the\n" + "# nested-code requirement, and the sealed resource directory then fails to\n" + "# parse). Entries here are tolerated; anything NOT here fails the check.\n" + "#\n" + "# This list should only ever shrink. Regenerate after renaming or deleting\n" + "# files with: python scripts/check_no_nonascii_asset_names.py --update-baseline\n" + "#\n" + "# Paths are where the file lands inside the .app payload. A few of them do\n" + "# not exist in the source tree because they ship inside an archive that is\n" + "# unpacked at build time (assets/*.tar.gz, frontend/pngtuber-packs/*.zip).\n" + ) + body = "".join(f"{entry}\n" for entry in sorted(offenders)) + path.write_text(header + body, encoding="utf-8") + + +def _explain(count: int) -> str: + return ( + f"\n{count} new non-ASCII bundled filename(s) found.\n" + "\n" + "Why this fails the build: Nuitka --mode=app puts the payload under\n" + "projectneko_server.app/Contents/MacOS/, and codesign treats everything\n" + "there as nested code. For each file it writes a requirement of the form\n" + "`identifier and anchor apple generic and ...` into CodeResources.\n" + "A non-ASCII name becomes a hex literal (`identifier 0xe4b883...`), which\n" + "is not valid requirement syntax, so the whole bundle then fails with\n" + "`the sealed resource directory is invalid` — blocking signing,\n" + "notarization and Steam upload. The error names no file, so this costs\n" + "hours to trace after the fact.\n" + "\n" + "CI cannot catch this: build-desktop.yml signs ad-hoc (`--sign -`), whose\n" + "requirements are `cdhash H\"...\"` and carry no identifier at all. Only the\n" + "local Developer ID path (build_mac.sh) hits it.\n" + "\n" + "Fix: give the file an ASCII name (stable slug or id) and update whatever\n" + "manifest references it. Do NOT add it to\n" + "scripts/nonascii_asset_baseline.txt — that list is a ratchet for assets\n" + "that predate this check and is only allowed to shrink.\n" + ) + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser( + description=( + "Fail on new non-ASCII filenames in assets bundled into the desktop app." + ) + ) + parser.add_argument( + "--include-untracked", + action="store_true", + help=( + "also walk the on-disk bundled roots (build outputs, unpacked models); " + "off by default so the result does not depend on build state" + ), + ) + parser.add_argument( + "--list", + action="store_true", + help="print every current offender (baselined included) and exit 0", + ) + parser.add_argument( + "--update-baseline", + action="store_true", + help="rewrite scripts/nonascii_asset_baseline.txt from the current tree", + ) + args = parser.parse_args(argv) + + offenders, from_archives = collect_offenders( + REPO_ROOT, include_untracked=args.include_untracked + ) + + if args.list: + for entry in sorted(offenders): + origin = from_archives.get(entry) + print(f"{entry}" + (f" (in {origin})" if origin else "")) + print(f"\n{len(offenders)} non-ASCII bundled path(s).") + return 0 + + if args.update_baseline: + if args.include_untracked: + # Untracked payload is build-state dependent; baking it into the + # baseline would make the file differ per machine. + print( + "error: --update-baseline refuses --include-untracked " + "(the result would depend on local build state)", + file=sys.stderr, + ) + return 2 + write_baseline(BASELINE_PATH, offenders) + rel = BASELINE_PATH.relative_to(REPO_ROOT).as_posix() + print(f"wrote {len(offenders)} entr(ies) to {rel}") + return 0 + + baseline = load_baseline(BASELINE_PATH) + new = sorted(offenders - baseline) + stale = sorted(baseline - offenders) + + for entry in new: + origin = from_archives.get(entry) + where = f" (ships inside {origin})" if origin else "" + print(f"{entry} {CODE} non-ASCII filename in bundled asset{where}") + + if stale and not new: + # Renamed or deleted — progress, never a failure. Nudge only, so a PR + # that legitimately removes an asset does not go red for it. + rel = BASELINE_PATH.relative_to(REPO_ROOT).as_posix() + print( + f"note: {len(stale)} baseline entr(ies) no longer exist — " + f"run `python scripts/check_no_nonascii_asset_names.py " + f"--update-baseline` to shrink {rel}:", + file=sys.stderr, + ) + for entry in stale[:10]: + print(f" - {entry}", file=sys.stderr) + if len(stale) > 10: + print(f" … and {len(stale) - 10} more", file=sys.stderr) + + if new: + print(_explain(len(new)), file=sys.stderr) + return 1 + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/nonascii_asset_baseline.txt b/scripts/nonascii_asset_baseline.txt new file mode 100644 index 0000000000..6e02c74707 --- /dev/null +++ b/scripts/nonascii_asset_baseline.txt @@ -0,0 +1,352 @@ +# Grandfathered non-ASCII filenames in bundled assets. +# +# See scripts/check_no_nonascii_asset_names.py for why these break macOS +# Developer ID signing (codesign emits a hex-literal `identifier` in the +# nested-code requirement, and the sealed resource directory then fails to +# parse). Entries here are tolerated; anything NOT here fails the check. +# +# This list should only ever shrink. Regenerate after renaming or deleting +# files with: python scripts/check_no_nonascii_asset_names.py --update-baseline +# +# Paths are where the file lands inside the .app payload. A few of them do +# not exist in the source tree because they ship inside an archive that is +# unpacked at build time (assets/*.tar.gz, frontend/pngtuber-packs/*.zip). +static/assets/tutorial/guide-audio/en/七天前，我们还只是第.mp3 +static/assets/tutorial/guide-audio/en/不管是想摸摸我的头，.mp3 +static/assets/tutorial/guide-audio/en/不管是说话的温度、相.mp3 +static/assets/tutorial/guide-audio/en/人家已经忍你很久了！.mp3 +static/assets/tutorial/guide-audio/en/今天带你认识的这些功.mp3 +static/assets/tutorial/guide-audio/en/今天的教程到这里就结.mp3 +static/assets/tutorial/guide-audio/en/今天，就让我悄悄跟上.mp3 +static/assets/tutorial/guide-audio/en/从今天起，我就真正成.mp3 +static/assets/tutorial/guide-audio/en/你可以放心地继续做你.mp3 +static/assets/tutorial/guide-audio/en/你可以随时来摸摸我的.mp3 +static/assets/tutorial/guide-audio/en/你要是计划有变，随时.mp3 +static/assets/tutorial/guide-audio/en/你选的每一个对话，都.mp3 +static/assets/tutorial/guide-audio/en/其实只要能这样陪着你.mp3 +static/assets/tutorial/guide-audio/en/前两天你一直在噼里啪.mp3 +static/assets/tutorial/guide-audio/en/呼……把这些繁琐的界.mp3 +static/assets/tutorial/guide-audio/en/咦，这里居然还能把我.mp3 +static/assets/tutorial/guide-audio/en/喵！现在是人家的教学.mp3 +static/assets/tutorial/guide-audio/en/嘻嘻，可别以为这个聊.mp3 +static/assets/tutorial/guide-audio/en/嘿嘿，前两天听到你的.mp3 +static/assets/tutorial/guide-audio/en/噔噔噔噔！今天必须要.mp3 +static/assets/tutorial/guide-audio/en/在跟我通语音电话的时.mp3 +static/assets/tutorial/guide-audio/en/在这个只属于我们的小.mp3 +static/assets/tutorial/guide-audio/en/在这个小按钮里，有许.mp3 +static/assets/tutorial/guide-audio/en/在这里可以决定我回复.mp3 +static/assets/tutorial/guide-audio/en/好啦好啦，不霸占你的.mp3 +static/assets/tutorial/guide-audio/en/好啦好啦，快去试试这.mp3 +static/assets/tutorial/guide-audio/en/如果你不小心忘记了我.mp3 +static/assets/tutorial/guide-audio/en/如果你想要看到更精致.mp3 +static/assets/tutorial/guide-audio/en/如果你现在需要专注、.mp3 +static/assets/tutorial/guide-audio/en/开启这个功能后，无论.mp3 +static/assets/tutorial/guide-audio/en/微风、阳光，还有刚刚.mp3 +static/assets/tutorial/guide-audio/en/微风还在窗边，阳光也.mp3 +static/assets/tutorial/guide-audio/en/快点开这个【Galg.mp3 +static/assets/tutorial/guide-audio/en/快让我也看看你眼前的.mp3 +static/assets/tutorial/guide-audio/en/快跟我老实交代，这两.mp3 +static/assets/tutorial/guide-audio/en/总是不小心触碰到、把.mp3 +static/assets/tutorial/guide-audio/en/我们不需要着急，每天.mp3 +static/assets/tutorial/guide-audio/en/戳一下聊天框上面的【.mp3 +static/assets/tutorial/guide-audio/en/把鼠标移到这里，长按.mp3 +static/assets/tutorial/guide-audio/en/最后警告一次喵！你要.mp3 +static/assets/tutorial/guide-audio/en/有了它们，我不光能看.mp3 +static/assets/tutorial/guide-audio/en/看这里看这里！当我决.mp3 +static/assets/tutorial/guide-audio/en/真是的，又在乱动鼠标.mp3 +static/assets/tutorial/guide-audio/en/真正舒服的陪伴才不是.mp3 +static/assets/tutorial/guide-audio/en/超级魔法开关出现！只.mp3 +static/assets/tutorial/guide-audio/en/这个小按钮也很重要哦.mp3 +static/assets/tutorial/guide-audio/en/这个是控制人家能不能.mp3 +static/assets/tutorial/guide-audio/en/这些小脚印，也可以由.mp3 +static/assets/tutorial/guide-audio/en/这里有一个神奇的按钮.mp3 +static/assets/tutorial/guide-audio/en/除了之前介绍的功能，.mp3 +static/assets/tutorial/guide-audio/ja/七天前，我们还只是第.mp3 +static/assets/tutorial/guide-audio/ja/不管是想摸摸我的头，.mp3 +static/assets/tutorial/guide-audio/ja/不管是说话的温度、相.mp3 +static/assets/tutorial/guide-audio/ja/人家已经忍你很久了！.mp3 +static/assets/tutorial/guide-audio/ja/今天带你认识的这些功.mp3 +static/assets/tutorial/guide-audio/ja/今天的教程到这里就结.mp3 +static/assets/tutorial/guide-audio/ja/今天，就让我悄悄跟上.mp3 +static/assets/tutorial/guide-audio/ja/从今天起，我就真正成.mp3 +static/assets/tutorial/guide-audio/ja/你可以放心地继续做你.mp3 +static/assets/tutorial/guide-audio/ja/你可以随时来摸摸我的.mp3 +static/assets/tutorial/guide-audio/ja/你要是计划有变，随时.mp3 +static/assets/tutorial/guide-audio/ja/你选的每一个对话，都.mp3 +static/assets/tutorial/guide-audio/ja/其实只要能这样陪着你.mp3 +static/assets/tutorial/guide-audio/ja/前两天你一直在噼里啪.mp3 +static/assets/tutorial/guide-audio/ja/呼……把这些繁琐的界.mp3 +static/assets/tutorial/guide-audio/ja/咦，这里居然还能把我.mp3 +static/assets/tutorial/guide-audio/ja/喵！现在是人家的教学.mp3 +static/assets/tutorial/guide-audio/ja/嘻嘻，可别以为这个聊.mp3 +static/assets/tutorial/guide-audio/ja/嘿嘿，前两天听到你的.mp3 +static/assets/tutorial/guide-audio/ja/噔噔噔噔！今天必须要.mp3 +static/assets/tutorial/guide-audio/ja/在跟我通语音电话的时.mp3 +static/assets/tutorial/guide-audio/ja/在这个只属于我们的小.mp3 +static/assets/tutorial/guide-audio/ja/在这个小按钮里，有许.mp3 +static/assets/tutorial/guide-audio/ja/在这里可以决定我回复.mp3 +static/assets/tutorial/guide-audio/ja/好啦好啦，不霸占你的.mp3 +static/assets/tutorial/guide-audio/ja/好啦好啦，快去试试这.mp3 +static/assets/tutorial/guide-audio/ja/如果你不小心忘记了我.mp3 +static/assets/tutorial/guide-audio/ja/如果你想要看到更精致.mp3 +static/assets/tutorial/guide-audio/ja/如果你现在需要专注、.mp3 +static/assets/tutorial/guide-audio/ja/开启这个功能后，无论.mp3 +static/assets/tutorial/guide-audio/ja/微风、阳光，还有刚刚.mp3 +static/assets/tutorial/guide-audio/ja/微风还在窗边，阳光也.mp3 +static/assets/tutorial/guide-audio/ja/快点开这个【Galg.mp3 +static/assets/tutorial/guide-audio/ja/快让我也看看你眼前的.mp3 +static/assets/tutorial/guide-audio/ja/快跟我老实交代，这两.mp3 +static/assets/tutorial/guide-audio/ja/总是不小心触碰到、把.mp3 +static/assets/tutorial/guide-audio/ja/我们不需要着急，每天.mp3 +static/assets/tutorial/guide-audio/ja/戳一下聊天框上面的【.mp3 +static/assets/tutorial/guide-audio/ja/把鼠标移到这里，长按.mp3 +static/assets/tutorial/guide-audio/ja/最后警告一次喵！你要.mp3 +static/assets/tutorial/guide-audio/ja/有了它们，我不光能看.mp3 +static/assets/tutorial/guide-audio/ja/看这里看这里！当我决.mp3 +static/assets/tutorial/guide-audio/ja/真是的，又在乱动鼠标.mp3 +static/assets/tutorial/guide-audio/ja/真正舒服的陪伴才不是.mp3 +static/assets/tutorial/guide-audio/ja/超级魔法开关出现！只.mp3 +static/assets/tutorial/guide-audio/ja/这个小按钮也很重要哦.mp3 +static/assets/tutorial/guide-audio/ja/这个是控制人家能不能.mp3 +static/assets/tutorial/guide-audio/ja/这些小脚印，也可以由.mp3 +static/assets/tutorial/guide-audio/ja/这里有一个神奇的按钮.mp3 +static/assets/tutorial/guide-audio/ja/除了之前介绍的功能，.mp3 +static/assets/tutorial/guide-audio/ko/七天前，我们还只是第.mp3 +static/assets/tutorial/guide-audio/ko/不管是想摸摸我的头，.mp3 +static/assets/tutorial/guide-audio/ko/不管是说话的温度、相.mp3 +static/assets/tutorial/guide-audio/ko/人家已经忍你很久了！.mp3 +static/assets/tutorial/guide-audio/ko/今天带你认识的这些功.mp3 +static/assets/tutorial/guide-audio/ko/今天的教程到这里就结.mp3 +static/assets/tutorial/guide-audio/ko/今天，就让我悄悄跟上.mp3 +static/assets/tutorial/guide-audio/ko/从今天起，我就真正成.mp3 +static/assets/tutorial/guide-audio/ko/你可以放心地继续做你.mp3 +static/assets/tutorial/guide-audio/ko/你可以随时来摸摸我的.mp3 +static/assets/tutorial/guide-audio/ko/你要是计划有变，随时.mp3 +static/assets/tutorial/guide-audio/ko/你选的每一个对话，都.mp3 +static/assets/tutorial/guide-audio/ko/其实只要能这样陪着你.mp3 +static/assets/tutorial/guide-audio/ko/前两天你一直在噼里啪.mp3 +static/assets/tutorial/guide-audio/ko/呼……把这些繁琐的界.mp3 +static/assets/tutorial/guide-audio/ko/咦，这里居然还能把我.mp3 +static/assets/tutorial/guide-audio/ko/喵！现在是人家的教学.mp3 +static/assets/tutorial/guide-audio/ko/嘻嘻，可别以为这个聊.mp3 +static/assets/tutorial/guide-audio/ko/嘿嘿，前两天听到你的.mp3 +static/assets/tutorial/guide-audio/ko/噔噔噔噔！今天必须要.mp3 +static/assets/tutorial/guide-audio/ko/在跟我通语音电话的时.mp3 +static/assets/tutorial/guide-audio/ko/在这个只属于我们的小.mp3 +static/assets/tutorial/guide-audio/ko/在这个小按钮里，有许.mp3 +static/assets/tutorial/guide-audio/ko/在这里可以决定我回复.mp3 +static/assets/tutorial/guide-audio/ko/好啦好啦，不霸占你的.mp3 +static/assets/tutorial/guide-audio/ko/好啦好啦，快去试试这.mp3 +static/assets/tutorial/guide-audio/ko/如果你不小心忘记了我.mp3 +static/assets/tutorial/guide-audio/ko/如果你想要看到更精致.mp3 +static/assets/tutorial/guide-audio/ko/如果你现在需要专注、.mp3 +static/assets/tutorial/guide-audio/ko/开启这个功能后，无论.mp3 +static/assets/tutorial/guide-audio/ko/微风、阳光，还有刚刚.mp3 +static/assets/tutorial/guide-audio/ko/微风还在窗边，阳光也.mp3 +static/assets/tutorial/guide-audio/ko/快点开这个【Galg.mp3 +static/assets/tutorial/guide-audio/ko/快让我也看看你眼前的.mp3 +static/assets/tutorial/guide-audio/ko/快跟我老实交代，这两.mp3 +static/assets/tutorial/guide-audio/ko/总是不小心触碰到、把.mp3 +static/assets/tutorial/guide-audio/ko/我们不需要着急，每天.mp3 +static/assets/tutorial/guide-audio/ko/戳一下聊天框上面的【.mp3 +static/assets/tutorial/guide-audio/ko/把鼠标移到这里，长按.mp3 +static/assets/tutorial/guide-audio/ko/最后警告一次喵！你要.mp3 +static/assets/tutorial/guide-audio/ko/有了它们，我不光能看.mp3 +static/assets/tutorial/guide-audio/ko/看这里看这里！当我决.mp3 +static/assets/tutorial/guide-audio/ko/真是的，又在乱动鼠标.mp3 +static/assets/tutorial/guide-audio/ko/真正舒服的陪伴才不是.mp3 +static/assets/tutorial/guide-audio/ko/超级魔法开关出现！只.mp3 +static/assets/tutorial/guide-audio/ko/这个小按钮也很重要哦.mp3 +static/assets/tutorial/guide-audio/ko/这个是控制人家能不能.mp3 +static/assets/tutorial/guide-audio/ko/这些小脚印，也可以由.mp3 +static/assets/tutorial/guide-audio/ko/这里有一个神奇的按钮.mp3 +static/assets/tutorial/guide-audio/ko/除了之前介绍的功能，.mp3 +static/assets/tutorial/guide-audio/ru/七天前，我们还只是第.mp3 +static/assets/tutorial/guide-audio/ru/不管是想摸摸我的头，.mp3 +static/assets/tutorial/guide-audio/ru/不管是说话的温度、相.mp3 +static/assets/tutorial/guide-audio/ru/人家已经忍你很久了！.mp3 +static/assets/tutorial/guide-audio/ru/今天带你认识的这些功.mp3 +static/assets/tutorial/guide-audio/ru/今天的教程到这里就结.mp3 +static/assets/tutorial/guide-audio/ru/今天，就让我悄悄跟上.mp3 +static/assets/tutorial/guide-audio/ru/从今天起，我就真正成.mp3 +static/assets/tutorial/guide-audio/ru/你可以放心地继续做你.mp3 +static/assets/tutorial/guide-audio/ru/你可以随时来摸摸我的.mp3 +static/assets/tutorial/guide-audio/ru/你要是计划有变，随时.mp3 +static/assets/tutorial/guide-audio/ru/你选的每一个对话，都.mp3 +static/assets/tutorial/guide-audio/ru/其实只要能这样陪着你.mp3 +static/assets/tutorial/guide-audio/ru/前两天你一直在噼里啪.mp3 +static/assets/tutorial/guide-audio/ru/呼……把这些繁琐的界.mp3 +static/assets/tutorial/guide-audio/ru/咦，这里居然还能把我.mp3 +static/assets/tutorial/guide-audio/ru/喵！现在是人家的教学.mp3 +static/assets/tutorial/guide-audio/ru/嘻嘻，可别以为这个聊.mp3 +static/assets/tutorial/guide-audio/ru/嘿嘿，前两天听到你的.mp3 +static/assets/tutorial/guide-audio/ru/噔噔噔噔！今天必须要.mp3 +static/assets/tutorial/guide-audio/ru/在跟我通语音电话的时.mp3 +static/assets/tutorial/guide-audio/ru/在这个只属于我们的小.mp3 +static/assets/tutorial/guide-audio/ru/在这个小按钮里，有许.mp3 +static/assets/tutorial/guide-audio/ru/在这里可以决定我回复.mp3 +static/assets/tutorial/guide-audio/ru/好啦好啦，不霸占你的.mp3 +static/assets/tutorial/guide-audio/ru/好啦好啦，快去试试这.mp3 +static/assets/tutorial/guide-audio/ru/如果你不小心忘记了我.mp3 +static/assets/tutorial/guide-audio/ru/如果你想要看到更精致.mp3 +static/assets/tutorial/guide-audio/ru/如果你现在需要专注、.mp3 +static/assets/tutorial/guide-audio/ru/开启这个功能后，无论.mp3 +static/assets/tutorial/guide-audio/ru/微风、阳光，还有刚刚.mp3 +static/assets/tutorial/guide-audio/ru/微风还在窗边，阳光也.mp3 +static/assets/tutorial/guide-audio/ru/快点开这个【Galg.mp3 +static/assets/tutorial/guide-audio/ru/快让我也看看你眼前的.mp3 +static/assets/tutorial/guide-audio/ru/快跟我老实交代，这两.mp3 +static/assets/tutorial/guide-audio/ru/总是不小心触碰到、把.mp3 +static/assets/tutorial/guide-audio/ru/我们不需要着急，每天.mp3 +static/assets/tutorial/guide-audio/ru/戳一下聊天框上面的【.mp3 +static/assets/tutorial/guide-audio/ru/把鼠标移到这里，长按.mp3 +static/assets/tutorial/guide-audio/ru/最后警告一次喵！你要.mp3 +static/assets/tutorial/guide-audio/ru/有了它们，我不光能看.mp3 +static/assets/tutorial/guide-audio/ru/看这里看这里！当我决.mp3 +static/assets/tutorial/guide-audio/ru/真是的，又在乱动鼠标.mp3 +static/assets/tutorial/guide-audio/ru/真正舒服的陪伴才不是.mp3 +static/assets/tutorial/guide-audio/ru/超级魔法开关出现！只.mp3 +static/assets/tutorial/guide-audio/ru/这个小按钮也很重要哦.mp3 +static/assets/tutorial/guide-audio/ru/这个是控制人家能不能.mp3 +static/assets/tutorial/guide-audio/ru/这些小脚印，也可以由.mp3 +static/assets/tutorial/guide-audio/ru/这里有一个神奇的按钮.mp3 +static/assets/tutorial/guide-audio/ru/除了之前介绍的功能，.mp3 +static/assets/tutorial/guide-audio/zh/七天前，我们还只是第.mp3 +static/assets/tutorial/guide-audio/zh/不管是想摸摸我的头，.mp3 +static/assets/tutorial/guide-audio/zh/不管是说话的温度、相.mp3 +static/assets/tutorial/guide-audio/zh/人家已经忍你很久了！.mp3 +static/assets/tutorial/guide-audio/zh/今天带你认识的这些功.mp3 +static/assets/tutorial/guide-audio/zh/今天的教程到这里就结.mp3 +static/assets/tutorial/guide-audio/zh/今天，就让我悄悄跟上.mp3 +static/assets/tutorial/guide-audio/zh/从今天起，我就真正成.mp3 +static/assets/tutorial/guide-audio/zh/你可以放心地继续做你.mp3 +static/assets/tutorial/guide-audio/zh/你可以随时来摸摸我的.mp3 +static/assets/tutorial/guide-audio/zh/你要是计划有变，随时.mp3 +static/assets/tutorial/guide-audio/zh/你选的每一个对话，都.mp3 +static/assets/tutorial/guide-audio/zh/其实只要能这样陪着你.mp3 +static/assets/tutorial/guide-audio/zh/前两天你一直在噼里啪.mp3 +static/assets/tutorial/guide-audio/zh/呼……把这些繁琐的界.mp3 +static/assets/tutorial/guide-audio/zh/咦，这里居然还能把我.mp3 +static/assets/tutorial/guide-audio/zh/喵！现在是人家的教学.mp3 +static/assets/tutorial/guide-audio/zh/嘻嘻，可别以为这个聊.mp3 +static/assets/tutorial/guide-audio/zh/嘿嘿，前两天听到你的.mp3 +static/assets/tutorial/guide-audio/zh/噔噔噔噔！今天必须要.mp3 +static/assets/tutorial/guide-audio/zh/在跟我通语音电话的时.mp3 +static/assets/tutorial/guide-audio/zh/在这个只属于我们的小.mp3 +static/assets/tutorial/guide-audio/zh/在这个小按钮里，有许.mp3 +static/assets/tutorial/guide-audio/zh/在这里可以决定我回复.mp3 +static/assets/tutorial/guide-audio/zh/好啦好啦，不霸占你的.mp3 +static/assets/tutorial/guide-audio/zh/好啦好啦，快去试试这.mp3 +static/assets/tutorial/guide-audio/zh/如果你不小心忘记了我.mp3 +static/assets/tutorial/guide-audio/zh/如果你想要看到更精致.mp3 +static/assets/tutorial/guide-audio/zh/如果你现在需要专注、.mp3 +static/assets/tutorial/guide-audio/zh/开启这个功能后，无论.mp3 +static/assets/tutorial/guide-audio/zh/微风、阳光，还有刚刚.mp3 +static/assets/tutorial/guide-audio/zh/微风还在窗边，阳光也.mp3 +static/assets/tutorial/guide-audio/zh/快点开这个【Galg.mp3 +static/assets/tutorial/guide-audio/zh/快让我也看看你眼前的.mp3 +static/assets/tutorial/guide-audio/zh/快跟我老实交代，这两.mp3 +static/assets/tutorial/guide-audio/zh/总是不小心触碰到、把.mp3 +static/assets/tutorial/guide-audio/zh/我们不需要着急，每天.mp3 +static/assets/tutorial/guide-audio/zh/戳一下聊天框上面的【.mp3 +static/assets/tutorial/guide-audio/zh/把鼠标移到这里，长按.mp3 +static/assets/tutorial/guide-audio/zh/最后警告一次喵！你要.mp3 +static/assets/tutorial/guide-audio/zh/有了它们，我不光能看.mp3 +static/assets/tutorial/guide-audio/zh/看这里看这里！当我决.mp3 +static/assets/tutorial/guide-audio/zh/真是的，又在乱动鼠标.mp3 +static/assets/tutorial/guide-audio/zh/真正舒服的陪伴才不是.mp3 +static/assets/tutorial/guide-audio/zh/超级魔法开关出现！只.mp3 +static/assets/tutorial/guide-audio/zh/这个小按钮也很重要哦.mp3 +static/assets/tutorial/guide-audio/zh/这个是控制人家能不能.mp3 +static/assets/tutorial/guide-audio/zh/这些小脚印，也可以由.mp3 +static/assets/tutorial/guide-audio/zh/这里有一个神奇的小按.mp3 +static/assets/tutorial/guide-audio/zh/这里有一个神奇的按钮.mp3 +static/assets/tutorial/guide-audio/zh/除了之前介绍的功能，.mp3 +static/game/games/soccer/audio/纯狐_心之所在_E.mp3 +static/game/games/soccer/audio/纯狐_心之所在_L.mp3 +static/game/games/soccer/audio/纯狐_心之所在_plus_E.mp3 +static/game/games/soccer/audio/纯狐_心之所在_plus_L.mp3 +static/mmd/animation/全身展示.vmd +static/mmd/animation/六亲不认.vmd +static/mmd/animation/射击姿态.vmd +static/mmd/animation/屈伸运动.vmd +static/mmd/animation/旋转.vmd +static/mmd/animation/模特姿势.vmd +static/mmd/animation/比V手势.vmd +static/mmd/animation/致意问候.vmd +static/mmd/animation/表情.vmd +static/mmd/animation/表情2.vmd +static/mmd/animation/表情3.vmd +static/mmd/animation/表情4.vmd +static/mmd/animation/表情5.vmd +static/mmd/animation/表情6.vmd +static/vrm/animation/全身展示.vrma.gz +static/vrm/animation/射击姿态.vrma.gz +static/vrm/animation/屈伸运动.vrma.gz +static/vrm/animation/旋转.vrma.gz +static/vrm/animation/模特姿势.vrma.gz +static/vrm/animation/比 V 手势.vrma.gz +static/vrm/animation/致意问候.vrma.gz +static/vrm/motion/01_base/夸张地晃动身体等待.vrma.gz +static/vrm/motion/01_base/尴尬或局促不安的来回看看身体.vrma.gz +static/vrm/motion/01_base/平静地左右环顾四周.vrma.gz +static/vrm/motion/01_base/开心地轻轻晃动等待.vrma.gz +static/vrm/motion/01_base/无聊地站着轻轻晃动.vrma.gz +static/vrm/motion/01_base/自然呼吸地安静站着.vrma.gz +static/vrm/motion/01_base/自然地站着等待.vrma.gz +static/vrm/motion/02_talk/伸手指向目标.vrma.gz +static/vrm/motion/02_talk/双手交替强硬反驳.vrma.gz +static/vrm/motion/02_talk/双手配合郑重表示赞同.vrma.gz +static/vrm/motion/02_talk/右手主导激烈争辩.vrma.gz +static/vrm/motion/02_talk/右手连续摆动表示不用.vrma.gz +static/vrm/motion/02_talk/坐着双手平稳细致解释.vrma.gz +static/vrm/motion/02_talk/坐着双手连续大幅交谈.vrma.gz +static/vrm/motion/02_talk/坚定地用力点头确认.vrma.gz +static/vrm/motion/02_talk/带着讽刺缓慢点头.vrma.gz +static/vrm/motion/02_talk/快速抬起右手简短认可.vrma.gz +static/vrm/motion/02_talk/生气地用力摇头拒绝.vrma.gz +static/vrm/motion/02_talk/站着双手展开认真说明.vrma.gz +static/vrm/motion/02_talk/站着右手主导温和解释.vrma.gz +static/vrm/motion/02_talk/站着右手小幅补充说明.vrma.gz +static/vrm/motion/02_talk/站着左手主导简短说明.vrma.gz +static/vrm/motion/02_talk/自然地摇头表示否定.vrma.gz +static/vrm/motion/02_talk/自然地点头表示同意.vrma.gz +static/vrm/motion/02_talk/若有所思地轻轻摇头.vrma.gz +static/vrm/motion/02_talk/若有所思地轻轻点头.vrma.gz +static/vrm/motion/02_talk/迅速抬起右手明确制止.vrma.gz +static/vrm/motion/03_emote/伤心地捂脸哭泣.vrma.gz +static/vrm/motion/03_emote/兴奋地大幅摆动身体.vrma.gz +static/vrm/motion/03_emote/受挫地垂下身体.vrma.gz +static/vrm/motion/03_emote/困倦地抬手打哈欠.vrma.gz +static/vrm/motion/03_emote/坐着开心地大笑.vrma.gz +static/vrm/motion/03_emote/失望地低头收拢身体.vrma.gz +static/vrm/motion/03_emote/害羞地把双手背在身后.vrma.gz +static/vrm/motion/03_emote/开心地舒展身体.vrma.gz +static/vrm/motion/03_emote/情绪过载地大幅挥动身体.vrma.gz +static/vrm/motion/03_emote/惊讶地向后缩起身体.vrma.gz +static/vrm/motion/03_emote/沮丧地低着头.vrma.gz +static/vrm/motion/03_emote/生气地绷紧身体和手臂.vrma.gz +static/vrm/motion/03_emote/站着开心地大笑.vrma.gz +static/vrm/motion/04_social/双手合十请求或道歉.vrma.gz +static/vrm/motion/04_social/坐着连续拍手鼓掌.vrma.gz +static/vrm/motion/04_social/夸张地举起双手欢呼.vrma.gz +static/vrm/motion/04_social/开心地举起双手挥手.vrma.gz +static/vrm/motion/04_social/快速随和地弯腰鞠躬.vrma.gz +static/vrm/motion/04_social/正式地抬手敬礼.vrma.gz +static/vrm/motion/04_social/站着连续拍手鼓掌.vrma.gz +static/vrm/motion/04_social/自然地举起一只手挥手.vrma.gz +static/vrm/motion/05_pose/休闲地盘着腿坐.vrma.gz +static/vrm/motion/05_pose/侧着身子慵懒地躺下.vrma.gz +static/vrm/motion/05_pose/双脚平放端正地坐.vrma.gz +static/vrm/motion/05_pose/嚣张地斜靠着半躺.vrma.gz +static/vrm/motion/05_pose/趴着安静熟睡轻微呼吸.vrma.gz +static/vrm/motion/05_pose/趴着睡梦中翻动身体.vrma.gz +static/vrm/motion/06_activity/坐着用双手弹奏钢琴.vrma.gz +static/vrm/motion/06_activity/坐着用双手敲击键盘.vrma.gz +static/vrm/motion/06_activity/站着用双手弹奏吉他.vrma.gz +static/vrm/motion/06_activity/站着用手操作平板电脑.vrma.gz +static/vrm/motion/07_show/持续踏步轻摆手臂跳嘻哈.vrma.gz +static/vrm/motion/07_show/柔和地跳伦巴舞.vrma.gz +static/vrm/motion/07_show/短促有力地大幅甩臂跳嘻哈.vrma.gz +static/vrm/motion/08_move/从低姿态撑起身体站好.vrma.gz +static/yui-origin/expressions/流汗.exp3.json diff --git a/tests/unit/test_check_no_nonascii_asset_names.py b/tests/unit/test_check_no_nonascii_asset_names.py new file mode 100644 index 0000000000..6911bd6c6d --- /dev/null +++ b/tests/unit/test_check_no_nonascii_asset_names.py @@ -0,0 +1,274 @@ +"""Unit tests for ``scripts/check_no_nonascii_asset_names.py``. + +Three-pronged coverage: + +1. Live-repo gate: run the real script against this tree. Exit must be 0. + This is what fails CI the moment someone lands a bundled asset with a + non-ASCII filename — the thing that silently breaks macOS Developer ID + signing with "the sealed resource directory is invalid". + +2. Detection logic: build a throwaway git repo plus a throwaway tar/zip and + assert the collector finds offenders on both paths (git listing and + archive members) while leaving ASCII names alone. + +3. Ratchet semantics: new offenders fail, baselined ones do not, and a + baseline entry whose file is gone is a note rather than a failure (so a + PR that legitimately deletes an asset does not go red). +""" +from __future__ import annotations + +import importlib.util +import subprocess +import sys +import tarfile +import zipfile +from pathlib import Path + +import pytest + +PROJECT_ROOT = Path(__file__).resolve().parents[2] +SCRIPT_PATH = PROJECT_ROOT / "scripts" / "check_no_nonascii_asset_names.py" +BASELINE_PATH = PROJECT_ROOT / "scripts" / "nonascii_asset_baseline.txt" + +# A real offender shape: truncated-transcript audio, the family that made +# this check necessary in the first place. +CJK_NAME = "七天前，我们.mp3" + + +def _load_script_module(): + """Import the checker as a module without ``scripts`` being a package.""" + spec = importlib.util.spec_from_file_location( + "check_no_nonascii_asset_names", SCRIPT_PATH, + ) + assert spec and spec.loader, f"failed to load spec for {SCRIPT_PATH}" + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +def _init_repo(root: Path) -> None: + """A minimal git repo — the checker asks git which files exist.""" + subprocess.run(["git", "init", "-q"], cwd=root, check=True) + + +# --------------------------------------------------------------------------- +# 1. Live-repo gate +# --------------------------------------------------------------------------- + + +@pytest.mark.unit +def test_live_repo_passes() -> None: + """Current tree must have no non-baselined non-ASCII bundled filename.""" + # 60s: the script is I/O bound (one `git ls-files`, a couple of archive + # listings) and normally finishes well under 5s. The cap keeps CI from + # hanging if an archive ever turns pathological. + result = subprocess.run( + [sys.executable, str(SCRIPT_PATH)], + cwd=str(PROJECT_ROOT), + capture_output=True, + text=True, + timeout=60, + ) + assert result.returncode == 0, ( + "new non-ASCII bundled asset filename(s) detected:\n" + f"stdout:\n{result.stdout}\nstderr:\n{result.stderr}" + ) + + +@pytest.mark.unit +def test_baseline_is_sorted_and_has_no_duplicates() -> None: + """The baseline is regenerated by --update-baseline; keep it canonical. + + A hand-edited, out-of-order or duplicated baseline produces noisy diffs + and makes "did this list shrink?" impossible to eyeball in review. + """ + entries = [ + line.strip() + for line in BASELINE_PATH.read_text(encoding="utf-8").splitlines() + if line.strip() and not line.strip().startswith("#") + ] + assert entries == sorted(entries), "baseline is not sorted" + assert len(entries) == len(set(entries)), "baseline has duplicate entries" + + +# --------------------------------------------------------------------------- +# 2. Detection logic +# --------------------------------------------------------------------------- + + +@pytest.mark.unit +def test_collects_offender_from_git_listing(tmp_path: Path) -> None: + """An untracked-but-not-ignored asset is caught before ``git add``.""" + module = _load_script_module() + _init_repo(tmp_path) + assets = tmp_path / "static" / "audio" + assets.mkdir(parents=True) + (assets / CJK_NAME).write_bytes(b"x") + (assets / "plain.mp3").write_bytes(b"x") + + offenders, from_archives = module.collect_offenders(tmp_path) + + assert offenders == {f"static/audio/{CJK_NAME}"} + assert from_archives == {} + + +@pytest.mark.unit +def test_non_ascii_directory_with_ascii_files_is_allowed(tmp_path: Path) -> None: + """Only basenames matter — codesign derives the identifier from those. + + Verified against a real Developer ID certificate: a bundle containing + ``MacOS//plain_name.png`` seals as ``identifier "plain_name"`` + and passes ``codesign --verify --deep --strict``. Flagging such paths + would be a failure nobody could act on. + """ + module = _load_script_module() + _init_repo(tmp_path) + directory = tmp_path / "static" / "素材目录" + directory.mkdir(parents=True) + (directory / "plain_name.png").write_bytes(b"x") + + offenders, _ = module.collect_offenders(tmp_path) + + assert offenders == set() + + +@pytest.mark.unit +def test_ignores_paths_outside_bundled_roots(tmp_path: Path) -> None: + """``tests/`` carries CJK fixture paths and never reaches the .app.""" + module = _load_script_module() + _init_repo(tmp_path) + outside = tmp_path / "tests" / "fixtures" + outside.mkdir(parents=True) + (outside / CJK_NAME).write_bytes(b"x") + + offenders, _ = module.collect_offenders(tmp_path) + + assert offenders == set() + + +@pytest.mark.unit +def test_ignores_gitignored_build_output(tmp_path: Path) -> None: + """Generated payload must not make the answer depend on build state.""" + module = _load_script_module() + _init_repo(tmp_path) + (tmp_path / ".gitignore").write_text("static/generated/\n", encoding="utf-8") + generated = tmp_path / "static" / "generated" + generated.mkdir(parents=True) + (generated / CJK_NAME).write_bytes(b"x") + + offenders, _ = module.collect_offenders(tmp_path) + assert offenders == set() + + # …but --include-untracked is exactly the "check what I just built" mode. + offenders_walked, _ = module.collect_offenders(tmp_path, include_untracked=True) + assert offenders_walked == {f"static/generated/{CJK_NAME}"} + + +@pytest.mark.unit +def test_collects_offender_inside_unpacked_archives(tmp_path: Path) -> None: + """Members of assets/*.tar.gz and pngtuber-packs/*.zip count as bundled. + + Both are expanded into ``static/`` at build time, so a non-ASCII member + reaches Contents/MacOS even though it never appears in the source tree. + Offenders are reported at their post-unpack destination. + """ + module = _load_script_module() + _init_repo(tmp_path) + + payload = tmp_path / "payload.bin" + payload.write_bytes(b"x") + + (tmp_path / "assets").mkdir() + with tarfile.open(tmp_path / "assets" / "model.tar.gz", "w:gz") as tar: + tar.add(payload, arcname=f"model/expressions/{CJK_NAME}") + tar.add(payload, arcname="model/model.moc3") + + packs = tmp_path / "frontend" / "pngtuber-packs" + packs.mkdir(parents=True) + with zipfile.ZipFile(packs / "tuber.zip", "w") as archive: + archive.writestr(f"tuber/layers/{CJK_NAME}", "x") + + offenders, from_archives = module.collect_offenders(tmp_path) + + assert offenders == { + f"static/model/expressions/{CJK_NAME}", + f"static/pngtuber/tuber/layers/{CJK_NAME}", + } + assert from_archives[f"static/model/expressions/{CJK_NAME}"] == ( + "assets/model.tar.gz" + ) + assert from_archives[f"static/pngtuber/tuber/layers/{CJK_NAME}"] == ( + "frontend/pngtuber-packs/tuber.zip" + ) + + +# --------------------------------------------------------------------------- +# 3. Ratchet semantics +# --------------------------------------------------------------------------- + + +def _run_in(root: Path, module, *argv: str) -> int: + """Run ``main`` with the module repointed at a throwaway tree.""" + module.REPO_ROOT = root + module.BASELINE_PATH = root / "scripts" / "nonascii_asset_baseline.txt" + return module.main(list(argv)) + + +@pytest.mark.unit +def test_new_offender_fails_but_baselined_one_passes( + tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + module = _load_script_module() + _init_repo(tmp_path) + (tmp_path / "scripts").mkdir() + assets = tmp_path / "static" / "audio" + assets.mkdir(parents=True) + (assets / CJK_NAME).write_bytes(b"x") + + # No baseline yet -> the file is new -> fail, with the codesign rationale + # spelled out rather than a bare path. + assert _run_in(tmp_path, module) == 1 + captured = capsys.readouterr() + assert f"static/audio/{CJK_NAME}" in captured.out + assert "NONASCII_ASSET_NAME" in captured.out + assert "sealed resource directory is invalid" in captured.err + + # Grandfather it, and the same tree is clean. + assert _run_in(tmp_path, module, "--update-baseline") == 0 + assert _run_in(tmp_path, module) == 0 + + # A second offender still fails even though the first is baselined. + (assets / f"another-{CJK_NAME}").write_bytes(b"x") + assert _run_in(tmp_path, module) == 1 + + +@pytest.mark.unit +def test_stale_baseline_entry_is_a_note_not_a_failure( + tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + """Deleting a grandfathered asset is progress; it must not go red.""" + module = _load_script_module() + _init_repo(tmp_path) + (tmp_path / "scripts").mkdir() + assets = tmp_path / "static" / "audio" + assets.mkdir(parents=True) + offender = assets / CJK_NAME + offender.write_bytes(b"x") + + assert _run_in(tmp_path, module, "--update-baseline") == 0 + offender.unlink() + + assert _run_in(tmp_path, module) == 0 + captured = capsys.readouterr() + assert "--update-baseline" in captured.err + assert f"static/audio/{CJK_NAME}" in captured.err + + +@pytest.mark.unit +def test_update_baseline_refuses_build_dependent_input(tmp_path: Path) -> None: + """A baseline seeded from local build output would differ per machine.""" + module = _load_script_module() + _init_repo(tmp_path) + (tmp_path / "scripts").mkdir() + + assert _run_in(tmp_path, module, "--update-baseline", "--include-untracked") == 2 From 907a91f758cac1bc9019388bab6562ba725393c9 Mon Sep 17 00:00:00 2001 From: "H.W." Date: Fri, 14 Aug 2026 14:09:02 -0700 Subject: [PATCH 02/13] =?UTF-8?q?fix(ci):=20=E5=A0=B5=E4=B8=8A=E6=A3=98?= =?UTF-8?q?=E8=BD=AE=E7=9A=84=E7=BB=95=E8=A1=8C=E5=8F=A3=EF=BC=8C=E5=B9=B6?= =?UTF-8?q?=E6=8A=8A=E6=89=AB=E6=8F=8F=E8=8C=83=E5=9B=B4=E5=AF=B9=E9=BD=90?= =?UTF-8?q?=E7=9C=9F=E5=AE=9E=E6=89=93=E5=8C=85=E9=9B=86=E5=90=88?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 评审的 6 条逐条核过，5 条成立，都在本次修掉： 1) 基线可以被撑大（CodeRabbit + Codex P1）。原来 --update-baseline 直接 按当前树重写，于是「加一个非 ASCII 资源 + 跑一次更新基线」就能全绿， 棘轮形同虚设。现在两头都堵：--update-baseline 只做交集，只会删掉已经 不存在的条目，遇到新 offender 直接报错退 1；新增 --base REF 把基线文件 本身与 merge-base 比对，多出任何一行即红——手写那行也跑不掉。CI 里按仓库 既有惯例挂成 PR-only 步骤，api-layering 的 checkout 相应改成 fetch-depth: 0。 2) PNGTuber 目的路径少了模型目录（Codex P2）。unpack_model 把 zip 解到 static/pngtuber//，原来记成 static/pngtuber/，路径是错的， 而且三个 pack 里同名成员会在字典里互相覆盖。改为从 manifest.json 逐条取 archive+folder，顺带解决了「只扫构建真正解压的档案」那条：不在 manifest 里的 zip 不再被扫。 3) tar 硬链接成员被漏掉（Codex P2）。TarInfo.isfile() 对硬链接为假，但 tar -xzm 会把它实打实落到 static/ 里。改为 isfile or islnk or issym。 4) docs/ 扫过头（Codex P2）。构建只打 docs/zh-CN/guide，扫整个 docs/ 会让 无关文档 PR 平白变红。范围收到该子树。 5) 语音模型目录不在扫描集合内（Codex P2）。endpointing/models 与 speaker_shadow/models 确实被 --include-data-dir 打进包。已加进 BUNDLED_ROOTS；.onnx 是构建时下载且 gitignore 的，git 列表看不见，只有 --include-untracked 的构建后扫描能覆盖，注释里写明了这个边界——按 manifest 预测下载文件名不做，太投机。 未采纳：assets/*.tar.gz 仍用 glob。当前两个 tar 恰好就是构建会解压的那两个， 而 glob 的失败方向是「多报」，比从 workflow 里反推解压列表更不易腐坏。 339 个 offender 数量不变，基线文件无需重算。新增 6 条测试覆盖：更新基线只能 缩短、基线相对 base ref 变长即红、base ref 上没有基线文件不算变长、tar 硬链接 成员、manifest 外的 zip 不扫、docs 只扫打包子树。16 条全绿，ruff 干净。 Co-Authored-By: Claude Opus 5 --- .github/workflows/analyze.yml | 19 ++ scripts/check_no_nonascii_asset_names.py | 173 ++++++++++++++++-- .../test_check_no_nonascii_asset_names.py | 168 ++++++++++++++++- 3 files changed, 336 insertions(+), 24 deletions(-) diff --git a/.github/workflows/analyze.yml b/.github/workflows/analyze.yml index 50cfce090c..ea3b3871c0 100644 --- a/.github/workflows/analyze.yml +++ b/.github/workflows/analyze.yml @@ -228,6 +228,11 @@ jobs: timeout-minutes: 10 steps: - uses: actions/checkout@v5 + with: + # The baseline-growth step below diffs scripts/nonascii_asset_baseline.txt + # against the merge-base, which a shallow clone cannot reach. Same + # trade the prompts-i18n job already makes: negligible on this repo. + fetch-depth: 0 - name: Set up Python 3.11 uses: actions/setup-python@v5 @@ -296,6 +301,20 @@ jobs: # tests/unit/test_check_no_nonascii_asset_names.py. run: python scripts/check_no_nonascii_asset_names.py + - name: Forbid growing the non-ASCII baseline (PR-only) + # The step above compares the tree against the baseline, so a PR that + # adds a non-ASCII asset AND its baseline line has an empty diff and + # passes — the ratchet would only be a convention. This step diffs the + # baseline itself against the merge-base and fails on any added entry, + # which is what makes "may only shrink" enforceable. `--update-baseline` + # already refuses to add; this catches the hand-written line too. + # Skipped on direct push to main — nothing to diff against. + if: github.event_name == 'pull_request' + env: + # Same zizmor template-injection indirection as the i18n-sync step. + BASE_REF: ${{ github.base_ref }} + run: python scripts/check_no_nonascii_asset_names.py --base "origin/${BASE_REF}" + core-contracts: name: Core package contracts runs-on: ubuntu-latest diff --git a/scripts/check_no_nonascii_asset_names.py b/scripts/check_no_nonascii_asset_names.py index 50a7e540e8..2f2fb1e2b5 100644 --- a/scripts/check_no_nonascii_asset_names.py +++ b/scripts/check_no_nonascii_asset_names.py @@ -65,10 +65,12 @@ ``git add`` rather than only after commit; - member names inside the archives that get unpacked into those roots at build time (``assets/*.tar.gz`` -> ``static//`` via - ``build_frontend.sh``; ``frontend/pngtuber-packs/*.zip`` -> - ``static/pngtuber//`` via ``scripts/unpack_builtin_pngtuber.py``). - Reading member names needs no build and no extraction, so the check gives - the same answer in a fresh checkout as on a build machine. + ``build_frontend.sh``; the PNGTuber packs named by + ``frontend/pngtuber-packs/manifest.json`` -> ``static/pngtuber//`` + via ``scripts/unpack_builtin_pngtuber.py``). Reading member names needs no + build and no extraction, so the check gives the same answer in a fresh + checkout as on a build machine. Hard-link and symlink members count too: + ``tar -xzm`` materializes them as entries under ``static/`` like any file. Asking git (rather than walking the working tree) keeps the result deterministic: gitignored build outputs, downloaded model weights, and local @@ -120,10 +122,18 @@ python scripts/check_no_nonascii_asset_names.py --list python scripts/check_no_nonascii_asset_names.py --include-untracked python scripts/check_no_nonascii_asset_names.py --update-baseline + python scripts/check_no_nonascii_asset_names.py --base origin/main + +The baseline is a ratchet in both directions of attack: ``--update-baseline`` +only ever drops entries that no longer exist, and ``--base`` fails when the +committed list gained a line relative to that ref. Without the second one the +first is only a convention — adding the asset and its baseline entry in the +same PR leaves nothing for the in-tree comparison to see. """ from __future__ import annotations import argparse +import json import os import subprocess import sys @@ -146,9 +156,19 @@ "templates", "assets", "data", - "docs", + # Not all of docs/ — the build packs exactly one subtree + # (--include-data-dir=docs/zh-CN/guide). Scanning the whole root would fail + # documentation PRs over files that never enter the payload. + "docs/zh-CN/guide", "frontend", "plugin/plugins", + # Voice-turn + speaker models, both --include-data-dir'd. The .onnx weights + # are downloaded at build time and gitignored, so the git listing cannot see + # them; these roots cover whatever *is* tracked here, and make + # --include-untracked reach the downloaded payload after a build. A manifest + # that names a non-ASCII .onnx is only caught by that post-build sweep. + "main_logic/asr_client/endpointing/models", + "main_logic/asr_client/speaker_shadow/models", ) # Archives that are expanded into a bundled root at build time. Value is the @@ -158,10 +178,14 @@ # build_frontend.sh: unpack_live2d_model -> static// ("assets", "static"), ) -ZIP_ARCHIVE_DESTS: tuple[tuple[str, str], ...] = ( - # scripts/unpack_builtin_pngtuber.py -> static/pngtuber// - ("frontend/pngtuber-packs", "static/pngtuber"), -) +# PNGTuber packs are not globbed: scripts/unpack_builtin_pngtuber.py unpacks +# exactly the archives listed in this manifest, and each one lands under its +# own `folder`. Both halves matter — globbing would flag a zip the build never +# opens, and a shared prefix would report `static/pngtuber/layers/x.png` for a +# member that really lands at `static/pngtuber/yui-origin/layers/x.png`, while +# collapsing same-named members from different packs into one entry. +ZIP_MANIFEST_REL = "frontend/pngtuber-packs/manifest.json" +ZIP_MANIFEST_DEST_ROOT = "static/pngtuber" # Never walked under --include-untracked. Vendored/derived trees whose names # we do not author; a hit in here is a bug in the upstream package, not in @@ -244,21 +268,61 @@ def _record(members: list[str], dest_prefix: str, archive_rel: str) -> None: continue for archive in sorted(directory.glob("*.tar.gz")): with tarfile.open(archive) as handle: - names = [m.name for m in handle.getmembers() if m.isfile()] + # islnk/issym as well as isfile: `tar -xzm` materializes hard + # links and symlinks as entries in static/ too, so a non-ASCII + # link name reaches the payload exactly like a regular file. + names = [ + m.name + for m in handle.getmembers() + if m.isfile() or m.islnk() or m.issym() + ] _record(names, dest_prefix, archive.relative_to(repo_root).as_posix()) - for source_dir, dest_prefix in ZIP_ARCHIVE_DESTS: - directory = repo_root / source_dir - if not directory.is_dir(): + for archive_rel, dest_prefix in _pngtuber_archive_dests(repo_root): + archive = repo_root / archive_rel + if not archive.is_file(): continue - for archive in sorted(directory.glob("*.zip")): - with zipfile.ZipFile(archive) as handle: - names = [i.filename for i in handle.infolist() if not i.is_dir()] - _record(names, dest_prefix, archive.relative_to(repo_root).as_posix()) + with zipfile.ZipFile(archive) as handle: + names = [i.filename for i in handle.infolist() if not i.is_dir()] + _record(names, dest_prefix, archive_rel) return offenders +def _pngtuber_archive_dests(repo_root: Path) -> list[tuple[str, str]]: + """(archive path, destination prefix) for each manifest-listed PNGTuber pack. + + Mirrors ``unpack_model``: the archive named by ``archive`` is expanded into + ``static/pngtuber//``. Entries missing either field are skipped — + the unpacker rejects them too, so they never reach the payload. + """ + manifest_path = repo_root / ZIP_MANIFEST_REL + if not manifest_path.is_file(): + return [] + try: + manifest = json.loads(manifest_path.read_text(encoding="utf-8")) + except (OSError, ValueError) as exc: + print(f"error: cannot read {ZIP_MANIFEST_REL}: {exc}", file=sys.stderr) + raise SystemExit(2) from exc + + packs_dir = PurePosixPath(ZIP_MANIFEST_REL).parent + dests: list[tuple[str, str]] = [] + for model in manifest.get("models", []): + if not isinstance(model, dict): + continue + folder = model.get("folder") + archive = model.get("archive") + if not isinstance(folder, str) or not isinstance(archive, str): + continue + dests.append( + ( + (packs_dir / archive).as_posix(), + f"{ZIP_MANIFEST_DEST_ROOT}/{folder}", + ) + ) + return sorted(dests) + + def _untracked_offenders(repo_root: Path) -> set[str]: """Non-ASCII paths found by walking the on-disk bundled roots. @@ -326,6 +390,33 @@ def write_baseline(path: Path, offenders: set[str]) -> None: path.write_text(header + body, encoding="utf-8") +def _baseline_growth(repo_root: Path, base_ref: str) -> list[str]: + """Baseline entries present now but absent at ``base_ref``. + + The in-tree comparison alone cannot enforce "only shrinks": a PR that adds + a non-ASCII asset *and* the matching baseline line has an empty + ``offenders - baseline`` and sails through. Only a diff against the merge + base sees that the list grew. + """ + rel = BASELINE_PATH.relative_to(repo_root).as_posix() + completed = subprocess.run( + ["git", "show", f"{base_ref}:{rel}"], + cwd=repo_root, + capture_output=True, + ) + if completed.returncode != 0: + # No baseline at the base ref — first landing of this check, or the + # ref is unreachable. Nothing to compare; the in-tree check still runs. + return [] + + before = { + line.strip() + for line in completed.stdout.decode("utf-8", errors="surrogateescape").splitlines() + if line.strip() and not line.strip().startswith("#") + } + return sorted(load_baseline(BASELINE_PATH) - before) + + def _explain(count: int) -> str: return ( f"\n{count} new non-ASCII bundled filename(s) found.\n" @@ -373,7 +464,18 @@ def main(argv: list[str] | None = None) -> int: parser.add_argument( "--update-baseline", action="store_true", - help="rewrite scripts/nonascii_asset_baseline.txt from the current tree", + help=( + "drop baseline entries that no longer exist; never adds (the list " + "is a ratchet)" + ), + ) + parser.add_argument( + "--base", + metavar="REF", + help=( + "also fail if the baseline gained entries relative to REF " + "(e.g. origin/main); PR-only, there is nothing to diff on a push" + ), ) args = parser.parse_args(argv) @@ -398,11 +500,42 @@ def main(argv: list[str] | None = None) -> int: file=sys.stderr, ) return 2 - write_baseline(BASELINE_PATH, offenders) + # Shrink-only. Writing `offenders` wholesale would let the documented + # workflow grandfather a brand-new violation: add the file, run this, + # and `offenders - baseline` comes back empty. Intersecting instead + # means the command can only ever drop entries that no longer exist. + existing = load_baseline(BASELINE_PATH) + kept = offenders & existing + rejected = sorted(offenders - existing) + write_baseline(BASELINE_PATH, kept) rel = BASELINE_PATH.relative_to(REPO_ROOT).as_posix() - print(f"wrote {len(offenders)} entr(ies) to {rel}") + print(f"wrote {len(kept)} entr(ies) to {rel} ({len(existing) - len(kept)} dropped)") + if rejected: + print( + f"\nerror: refusing to add {len(rejected)} new entr(ies) — the " + "baseline may only shrink:", + file=sys.stderr, + ) + for entry in rejected: + print(f" - {entry}", file=sys.stderr) + print(_explain(len(rejected)), file=sys.stderr) + return 1 return 0 + if args.base: + grown = _baseline_growth(REPO_ROOT, args.base) + if grown: + rel = BASELINE_PATH.relative_to(REPO_ROOT).as_posix() + print( + f"{rel} {CODE} {len(grown)} entr(ies) added to the baseline " + f"since {args.base}:", + file=sys.stderr, + ) + for entry in grown: + print(f" + {entry}", file=sys.stderr) + print(_explain(len(grown)), file=sys.stderr) + return 1 + baseline = load_baseline(BASELINE_PATH) new = sorted(offenders - baseline) stale = sorted(baseline - offenders) diff --git a/tests/unit/test_check_no_nonascii_asset_names.py b/tests/unit/test_check_no_nonascii_asset_names.py index 6911bd6c6d..10f5168f05 100644 --- a/tests/unit/test_check_no_nonascii_asset_names.py +++ b/tests/unit/test_check_no_nonascii_asset_names.py @@ -18,6 +18,7 @@ from __future__ import annotations import importlib.util +import json import subprocess import sys import tarfile @@ -186,7 +187,13 @@ def test_collects_offender_inside_unpacked_archives(tmp_path: Path) -> None: packs = tmp_path / "frontend" / "pngtuber-packs" packs.mkdir(parents=True) with zipfile.ZipFile(packs / "tuber.zip", "w") as archive: - archive.writestr(f"tuber/layers/{CJK_NAME}", "x") + archive.writestr(f"layers/{CJK_NAME}", "x") + # unpack_builtin_pngtuber.py drives off this manifest: each listed archive + # is expanded under its own `folder`, which is where members really land. + (packs / "manifest.json").write_text( + json.dumps({"models": [{"folder": "tuber", "archive": "tuber.zip"}]}), + encoding="utf-8", + ) offenders, from_archives = module.collect_offenders(tmp_path) @@ -207,6 +214,16 @@ def test_collects_offender_inside_unpacked_archives(tmp_path: Path) -> None: # --------------------------------------------------------------------------- +def _write_baseline(root: Path, entries: list[str]) -> None: + """Seed a grandfathered baseline directly, the way a human would.""" + path = root / "scripts" / "nonascii_asset_baseline.txt" + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text( + "# grandfathered\n" + "".join(f"{e}\n" for e in sorted(entries)), + encoding="utf-8", + ) + + def _run_in(root: Path, module, *argv: str) -> int: """Run ``main`` with the module repointed at a throwaway tree.""" module.REPO_ROOT = root @@ -233,8 +250,9 @@ def test_new_offender_fails_but_baselined_one_passes( assert "NONASCII_ASSET_NAME" in captured.out assert "sealed resource directory is invalid" in captured.err - # Grandfather it, and the same tree is clean. - assert _run_in(tmp_path, module, "--update-baseline") == 0 + # Grandfathering is a deliberate, hand-written act — --update-baseline + # refuses to do it (see test_update_baseline_only_ever_shrinks). + _write_baseline(tmp_path, [f"static/audio/{CJK_NAME}"]) assert _run_in(tmp_path, module) == 0 # A second offender still fails even though the first is baselined. @@ -255,7 +273,7 @@ def test_stale_baseline_entry_is_a_note_not_a_failure( offender = assets / CJK_NAME offender.write_bytes(b"x") - assert _run_in(tmp_path, module, "--update-baseline") == 0 + _write_baseline(tmp_path, [f"static/audio/{CJK_NAME}"]) offender.unlink() assert _run_in(tmp_path, module) == 0 @@ -272,3 +290,145 @@ def test_update_baseline_refuses_build_dependent_input(tmp_path: Path) -> None: (tmp_path / "scripts").mkdir() assert _run_in(tmp_path, module, "--update-baseline", "--include-untracked") == 2 + + +@pytest.mark.unit +def test_update_baseline_only_ever_shrinks( + tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + """The documented command must not be a way to grandfather a new asset. + + Rewriting the baseline from the current tree would make `offenders - + baseline` empty for a file added in the same PR, turning the ratchet into + a rubber stamp. + """ + module = _load_script_module() + _init_repo(tmp_path) + assets = tmp_path / "static" / "audio" + assets.mkdir(parents=True) + (assets / CJK_NAME).write_bytes(b"x") + gone = f"static/audio/deleted-{CJK_NAME}" + _write_baseline(tmp_path, [gone]) + + assert _run_in(tmp_path, module, "--update-baseline") == 1 + captured = capsys.readouterr() + assert "may only shrink" in captured.err + assert f"static/audio/{CJK_NAME}" in captured.err + + # The vanished entry is still dropped — shrinking is the whole point — + # but the new offender was not written, so the tree stays red. + baseline = module.load_baseline(tmp_path / "scripts" / "nonascii_asset_baseline.txt") + assert baseline == set() + assert _run_in(tmp_path, module) == 1 + + +@pytest.mark.unit +def test_baseline_growth_against_base_ref_fails( + tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + """Hand-editing the baseline is caught by the diff against the merge base. + + --update-baseline refusing to add is only half the ratchet; a contributor + can still type the line by hand. Comparing with the base ref is what makes + "only shrinks" actually true. + """ + module = _load_script_module() + _init_repo(tmp_path) + assets = tmp_path / "static" / "audio" + assets.mkdir(parents=True) + _write_baseline(tmp_path, []) + subprocess.run(["git", "add", "-A"], cwd=tmp_path, check=True) + subprocess.run( + ["git", "-c", "user.email=t@t", "-c", "user.name=t", "commit", "-qm", "base"], + cwd=tmp_path, + check=True, + ) + + # Add the asset AND the matching baseline line — the in-tree comparison + # alone sees nothing wrong. + (assets / CJK_NAME).write_bytes(b"x") + _write_baseline(tmp_path, [f"static/audio/{CJK_NAME}"]) + assert _run_in(tmp_path, module) == 0 + + assert _run_in(tmp_path, module, "--base", "HEAD") == 1 + captured = capsys.readouterr() + assert f"+ static/audio/{CJK_NAME}" in captured.err + + # Shrinking against the same base ref stays green. + _write_baseline(tmp_path, []) + assert _run_in(tmp_path, module, "--base", "HEAD") == 1 # asset now unbaselined + (assets / CJK_NAME).unlink() + assert _run_in(tmp_path, module, "--base", "HEAD") == 0 + + +@pytest.mark.unit +def test_missing_baseline_at_base_ref_is_not_growth(tmp_path: Path) -> None: + """First landing of this check: the base ref has no baseline to diff.""" + module = _load_script_module() + _init_repo(tmp_path) + (tmp_path / "static").mkdir() + (tmp_path / "static" / "ok.txt").write_bytes(b"x") + subprocess.run(["git", "add", "-A"], cwd=tmp_path, check=True) + subprocess.run( + ["git", "-c", "user.email=t@t", "-c", "user.name=t", "commit", "-qm", "base"], + cwd=tmp_path, + check=True, + ) + _write_baseline(tmp_path, ["static/audio/whatever.mp3"]) + + assert _run_in(tmp_path, module, "--base", "HEAD") == 0 + + +@pytest.mark.unit +def test_tar_hard_link_member_counts_as_extracted(tmp_path: Path) -> None: + """`tar -xzm` materializes hard links, so their names reach the payload. + + TarInfo.isfile() is False for a hard-link entry, which used to hide a + non-ASCII name that the build nevertheless writes into static/. + """ + module = _load_script_module() + _init_repo(tmp_path) + payload = tmp_path / "payload.bin" + payload.write_bytes(b"x") + + (tmp_path / "assets").mkdir() + with tarfile.open(tmp_path / "assets" / "model.tar.gz", "w:gz") as tar: + tar.add(payload, arcname="model/real.png") + link = tarfile.TarInfo(f"model/{CJK_NAME}") + link.type = tarfile.LNKTYPE + link.linkname = "model/real.png" + tar.addfile(link) + + offenders, _ = module.collect_offenders(tmp_path) + assert offenders == {f"static/model/{CJK_NAME}"} + + +@pytest.mark.unit +def test_zip_outside_the_pngtuber_manifest_is_ignored(tmp_path: Path) -> None: + """Only manifest-listed packs are unpacked; the rest ship opaque.""" + module = _load_script_module() + _init_repo(tmp_path) + packs = tmp_path / "frontend" / "pngtuber-packs" + packs.mkdir(parents=True) + with zipfile.ZipFile(packs / "unused.zip", "w") as archive: + archive.writestr(f"layers/{CJK_NAME}", "x") + (packs / "manifest.json").write_text(json.dumps({"models": []}), encoding="utf-8") + + offenders, _ = module.collect_offenders(tmp_path) + assert offenders == set() + + +@pytest.mark.unit +def test_only_the_packaged_docs_subtree_is_scanned(tmp_path: Path) -> None: + """The build packs docs/zh-CN/guide, not docs/ — no false positives.""" + module = _load_script_module() + _init_repo(tmp_path) + packaged = tmp_path / "docs" / "zh-CN" / "guide" + packaged.mkdir(parents=True) + (packaged / f"{CJK_NAME}.md").write_bytes(b"x") + elsewhere = tmp_path / "docs" / "design" + elsewhere.mkdir(parents=True) + (elsewhere / f"{CJK_NAME}.md").write_bytes(b"x") + + offenders, _ = module.collect_offenders(tmp_path) + assert offenders == {f"docs/zh-CN/guide/{CJK_NAME}.md"} From ff13ca1b2bcfb9319c5b92ddfdb767536b00af0a Mon Sep 17 00:00:00 2001 From: "H.W." Date: Fri, 14 Aug 2026 14:26:28 -0700 Subject: [PATCH 03/13] =?UTF-8?q?fix(ci):=20=E6=94=B6=E7=AA=84=20frontend?= =?UTF-8?q?=20=E6=89=AB=E6=8F=8F=E9=9D=A2=EF=BC=8C=E6=94=B9=E4=BB=8E=20man?= =?UTF-8?q?ifest=20=E7=9B=B4=E6=8E=A5=E6=9F=A5=E4=B8=8B=E8=BD=BD=E6=A8=A1?= =?UTF-8?q?=E5=9E=8B=E5=90=8D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 第二轮 6 条，核过 4 条成立： 1) frontend/ 扫过头（Codex P2）。三个构建脚本（build-desktop.yml、 build_nuitka.bat、build_mac.sh）都只打 frontend/plugin-manager/dist，前端 源码不进 payload，React 产物走 static/。原来扫整个 frontend/ 会因为一个 frontend/plugin-manager/src/组件.vue 把无关 PR 打红。范围收到 dist。 2) 语音模型名在 CI 里查不到（Codex P2，上一轮我按"太投机"回绝了，这轮的论据 成立）：.onnx 是 gitignore 的下载物，而 analyze.yml 不带 --include-untracked， 于是上一轮新加的两个 roots 在 CI 里等于没用。但文件名本身是提交进仓库的—— 两个 manifest.json 里的 filename 字段。改为直接读这两个 manifest 校验 filename，两种 schema（顶层 filename、assets[].filename）都覆盖，不需要构建。 3) --base 引用无法解析时静默放行（CodeRabbit Major）。原来把"base 上没有基线 文件"和"引用根本不存在"一视同仁地当作没增长，于是浅克隆没 fetch origin/main、 或者引用拼错，棘轮就在最该生效的时候变绿。改为先 git rev-parse --verify 校验 引用可解析，不可解析直接 SystemExit(2)；只有引用有效但基线文件不存在才算 首次落地。 4) manifest 结构非法时静默变成空扫描（CodeRabbit Minor）。顶层是数组/字符串时 .get 会抛 AttributeError，models 不是列表时又会静默跳过。两处都改成带信息的 SystemExit(2)——"没找到 pack"和"manifest 是个数组"不该长得一样。 驳回 2 条（CodeRabbit Major + 配套测试建议）：要求把 bin、vendor/openfang、src 加进 BUNDLED_ROOTS，理由是"Electron Builder 会发布这些路径、Forge 会暂存"。本 仓库根本没有这三个目录，也没有 package.json 或 forge.config.js——那是 N.E.K.O.-PC （Electron 端）的结构，评审把两个仓库搞混了。加进去只会是三条死配置。 339 个 offender 数量不变，基线无需重算。新增 4 条测试（manifest filename 两种 schema、非法 manifest 报错、base 引用不可解析报错、frontend 只扫 dist），20 条 全绿，ruff 干净。 Co-Authored-By: Claude Opus 5 --- scripts/check_no_nonascii_asset_names.py | 112 ++++++++++++++++-- .../test_check_no_nonascii_asset_names.py | 85 +++++++++++++ 2 files changed, 185 insertions(+), 12 deletions(-) diff --git a/scripts/check_no_nonascii_asset_names.py b/scripts/check_no_nonascii_asset_names.py index 2f2fb1e2b5..3b3524d2c5 100644 --- a/scripts/check_no_nonascii_asset_names.py +++ b/scripts/check_no_nonascii_asset_names.py @@ -63,6 +63,10 @@ - files git knows about under the bundled roots (``BUNDLED_ROOTS``) — tracked plus untracked-but-not-ignored, so the check fires before ``git add`` rather than only after commit; +- ``filename`` values in the manifests that drive build-time downloads + (``main_logic/asr_client/*/models/manifest.json``): the ``.onnx`` itself is + gitignored and only exists after a build, but the name it will be written + under is committed, so a plain CI run can still catch it; - member names inside the archives that get unpacked into those roots at build time (``assets/*.tar.gz`` -> ``static//`` via ``build_frontend.sh``; the PNGTuber packs named by @@ -160,17 +164,32 @@ # (--include-data-dir=docs/zh-CN/guide). Scanning the whole root would fail # documentation PRs over files that never enter the payload. "docs/zh-CN/guide", - "frontend", + # Only the vite output ships (--include-data-dir=frontend/plugin-manager/dist, + # identically in build_nuitka.bat, build_mac.sh and build-desktop.yml). + # Frontend *sources* never enter the payload — React's output goes to + # static/ — so scanning all of frontend/ would fail a PR over a file that + # cannot possibly break signing. + "frontend/plugin-manager/dist", "plugin/plugins", # Voice-turn + speaker models, both --include-data-dir'd. The .onnx weights # are downloaded at build time and gitignored, so the git listing cannot see # them; these roots cover whatever *is* tracked here, and make - # --include-untracked reach the downloaded payload after a build. A manifest - # that names a non-ASCII .onnx is only caught by that post-build sweep. + # --include-untracked reach the downloaded payload after a build. The names + # the downloader will write are checked from the manifests below, so a plain + # CI run catches them without building. "main_logic/asr_client/endpointing/models", "main_logic/asr_client/speaker_shadow/models", ) +# Manifests naming files the build downloads into a bundled root. The weights are +# gitignored, so no git listing can see them before a build — but the name is +# committed right here, so it can be checked in a plain run. ``filename`` sits +# either at the top level or inside an ``assets`` list, depending on the schema. +MODEL_MANIFESTS: tuple[str, ...] = ( + "main_logic/asr_client/endpointing/models/manifest.json", + "main_logic/asr_client/speaker_shadow/models/manifest.json", +) + # Archives that are expanded into a bundled root at build time. Value is the # bundled path prefix the members land under, so a violation reports where the # file will actually sit inside the .app rather than where it hides today. @@ -289,6 +308,47 @@ def _record(members: list[str], dest_prefix: str, archive_rel: str) -> None: return offenders +def _load_json(path: Path, rel: str) -> object: + try: + return json.loads(path.read_text(encoding="utf-8")) + except (OSError, ValueError) as exc: + print(f"error: cannot read {rel}: {exc}", file=sys.stderr) + raise SystemExit(2) from exc + + +def _model_manifest_offenders(repo_root: Path) -> dict[str, str]: + """Non-ASCII ``filename`` values in the downloaded-model manifests. + + Keyed by where the download lands, valued by the manifest to edit. This is + what makes the two model roots useful in CI: the payload itself only exists + after a build, but the name that will be written is committed. + """ + offenders: dict[str, str] = {} + for rel in MODEL_MANIFESTS: + path = repo_root / rel + if not path.is_file(): + continue + manifest = _load_json(path, rel) + if not isinstance(manifest, dict): + print(f"error: invalid {rel}: top level must be an object", file=sys.stderr) + raise SystemExit(2) + + entries: list[dict] = [manifest] + assets = manifest.get("assets") + if assets is not None: + if not isinstance(assets, list): + print(f"error: invalid {rel}: 'assets' must be a list", file=sys.stderr) + raise SystemExit(2) + entries.extend(item for item in assets if isinstance(item, dict)) + + dest_dir = PurePosixPath(rel).parent.as_posix() + for entry in entries: + name = entry.get("filename") + if isinstance(name, str) and name and not _is_ascii(PurePosixPath(name).name): + offenders[f"{dest_dir}/{name}"] = rel + return offenders + + def _pngtuber_archive_dests(repo_root: Path) -> list[tuple[str, str]]: """(archive path, destination prefix) for each manifest-listed PNGTuber pack. @@ -299,15 +359,26 @@ def _pngtuber_archive_dests(repo_root: Path) -> list[tuple[str, str]]: manifest_path = repo_root / ZIP_MANIFEST_REL if not manifest_path.is_file(): return [] - try: - manifest = json.loads(manifest_path.read_text(encoding="utf-8")) - except (OSError, ValueError) as exc: - print(f"error: cannot read {ZIP_MANIFEST_REL}: {exc}", file=sys.stderr) - raise SystemExit(2) from exc + manifest = _load_json(manifest_path, ZIP_MANIFEST_REL) + # A malformed manifest must be a loud error, not a silently empty scan — + # "no packs found" and "the file is a list" would otherwise look identical. + if not isinstance(manifest, dict): + print( + f"error: invalid {ZIP_MANIFEST_REL}: top level must be an object", + file=sys.stderr, + ) + raise SystemExit(2) + models = manifest.get("models", []) + if not isinstance(models, list): + print( + f"error: invalid {ZIP_MANIFEST_REL}: 'models' must be a list", + file=sys.stderr, + ) + raise SystemExit(2) packs_dir = PurePosixPath(ZIP_MANIFEST_REL).parent dests: list[tuple[str, str]] = [] - for model in manifest.get("models", []): + for model in models: if not isinstance(model, dict): continue folder = model.get("folder") @@ -350,8 +421,9 @@ def _untracked_offenders(repo_root: Path) -> set[str]: def collect_offenders( repo_root: Path, include_untracked: bool = False ) -> tuple[set[str], dict[str, str]]: - """Return (bundled paths with non-ASCII names, path -> owning archive).""" + """Return (bundled paths with non-ASCII names, path -> owning archive/manifest).""" from_archives = _archive_offenders(repo_root) + from_archives.update(_model_manifest_offenders(repo_root)) offenders = _git_listed_offenders(repo_root) | set(from_archives) if include_untracked: offenders |= _untracked_offenders(repo_root) @@ -398,6 +470,22 @@ def _baseline_growth(repo_root: Path, base_ref: str) -> list[str]: ``offenders - baseline`` and sails through. Only a diff against the merge base sees that the list grew. """ + # Resolve the ref first. Without this, an unreachable ref (shallow clone + # that never fetched origin/main, a typo, a renamed default branch) is + # indistinguishable from "the baseline did not exist yet" — and the ratchet + # would silently pass exactly when it is needed. Fail loudly instead. + if subprocess.run( + ["git", "rev-parse", "--verify", "--quiet", f"{base_ref}^{{commit}}"], + cwd=repo_root, + capture_output=True, + ).returncode != 0: + print( + f"error: --base {base_ref} does not resolve to a commit " + "(shallow clone, or the ref was never fetched)", + file=sys.stderr, + ) + raise SystemExit(2) + rel = BASELINE_PATH.relative_to(repo_root).as_posix() completed = subprocess.run( ["git", "show", f"{base_ref}:{rel}"], @@ -405,8 +493,8 @@ def _baseline_growth(repo_root: Path, base_ref: str) -> list[str]: capture_output=True, ) if completed.returncode != 0: - # No baseline at the base ref — first landing of this check, or the - # ref is unreachable. Nothing to compare; the in-tree check still runs. + # The ref is good but carries no baseline: first landing of this check. + # Nothing to compare; the in-tree check still runs. return [] before = { diff --git a/tests/unit/test_check_no_nonascii_asset_names.py b/tests/unit/test_check_no_nonascii_asset_names.py index 10f5168f05..166c0f0d72 100644 --- a/tests/unit/test_check_no_nonascii_asset_names.py +++ b/tests/unit/test_check_no_nonascii_asset_names.py @@ -432,3 +432,88 @@ def test_only_the_packaged_docs_subtree_is_scanned(tmp_path: Path) -> None: offenders, _ = module.collect_offenders(tmp_path) assert offenders == {f"docs/zh-CN/guide/{CJK_NAME}.md"} + + +@pytest.mark.unit +def test_model_manifest_filename_is_checked_without_a_build(tmp_path: Path) -> None: + """The .onnx weights are gitignored; the name that will be written is not. + + Both manifest shapes are covered: `filename` at the top level and inside an + `assets` list. Without this, the two model roots would only pay off under + --include-untracked, which CI never runs. + """ + module = _load_script_module() + _init_repo(tmp_path) + for rel in module.MODEL_MANIFESTS: + (tmp_path / rel).parent.mkdir(parents=True, exist_ok=True) + top_level, nested = module.MODEL_MANIFESTS[1], module.MODEL_MANIFESTS[0] + (tmp_path / top_level).write_text( + json.dumps({"filename": f"{CJK_NAME}.onnx"}), encoding="utf-8" + ) + (tmp_path / nested).write_text( + json.dumps({"assets": [{"filename": "ascii.onnx"}, {"filename": f"x{CJK_NAME}.onnx"}]}), + encoding="utf-8", + ) + + offenders, from_archives = module.collect_offenders(tmp_path) + + top_dir = str(Path(top_level).parent).replace("\\", "/") + nested_dir = str(Path(nested).parent).replace("\\", "/") + assert offenders == { + f"{top_dir}/{CJK_NAME}.onnx", + f"{nested_dir}/x{CJK_NAME}.onnx", + } + assert from_archives[f"{top_dir}/{CJK_NAME}.onnx"] == top_level + + +@pytest.mark.unit +def test_malformed_manifest_is_a_loud_error(tmp_path: Path) -> None: + """"No packs found" and "the manifest is a list" must not look the same.""" + module = _load_script_module() + _init_repo(tmp_path) + packs = tmp_path / "frontend" / "pngtuber-packs" + packs.mkdir(parents=True) + (packs / "manifest.json").write_text(json.dumps([{"folder": "x"}]), encoding="utf-8") + + with pytest.raises(SystemExit) as excinfo: + module.collect_offenders(tmp_path) + assert excinfo.value.code == 2 + + (packs / "manifest.json").write_text(json.dumps({"models": "nope"}), encoding="utf-8") + with pytest.raises(SystemExit) as excinfo: + module.collect_offenders(tmp_path) + assert excinfo.value.code == 2 + + +@pytest.mark.unit +def test_unresolvable_base_ref_fails_instead_of_passing(tmp_path: Path) -> None: + """A ref that cannot be resolved must not read as "the baseline didn't grow". + + Otherwise a shallow clone that never fetched origin/main turns the ratchet + green exactly when it is supposed to bite. + """ + module = _load_script_module() + _init_repo(tmp_path) + _write_baseline(tmp_path, ["static/audio/whatever.mp3"]) + + # SystemExit(2), like the other environment failures in this script + # (unreadable git listing, malformed manifest) — not a lint verdict. + with pytest.raises(SystemExit) as excinfo: + _run_in(tmp_path, module, "--base", "origin/nope") + assert excinfo.value.code == 2 + + +@pytest.mark.unit +def test_only_the_packaged_frontend_output_is_scanned(tmp_path: Path) -> None: + """All three build scripts pack frontend/plugin-manager/dist and nothing else.""" + module = _load_script_module() + _init_repo(tmp_path) + source = tmp_path / "frontend" / "plugin-manager" / "src" + source.mkdir(parents=True) + (source / f"{CJK_NAME}.vue").write_bytes(b"x") + packaged = tmp_path / "frontend" / "plugin-manager" / "dist" / "assets" + packaged.mkdir(parents=True) + (packaged / f"{CJK_NAME}.js").write_bytes(b"x") + + offenders, _ = module.collect_offenders(tmp_path) + assert offenders == {f"frontend/plugin-manager/dist/assets/{CJK_NAME}.js"} From 0e5ca14f4b83f00074a01e2fce0390b3804a883d Mon Sep 17 00:00:00 2001 From: "H.W." Date: Fri, 14 Aug 2026 14:42:04 -0700 Subject: [PATCH 04/13] =?UTF-8?q?fix(ci):=20=E6=89=AB=E6=8F=8F=E9=9D=A2?= =?UTF-8?q?=E5=AF=B9=E9=BD=90=E7=9C=9F=E5=AE=9E=E6=89=93=E5=8C=85=E9=9B=86?= =?UTF-8?q?=E5=90=88=E7=9A=84=E7=AC=AC=E4=B8=89=E8=BD=AE=EF=BC=88=E5=9B=9B?= =?UTF-8?q?=E6=9D=A1=E5=85=A8=E9=83=A8=E6=88=90=E7=AB=8B=EF=BC=89?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 1) vite public 漏扫。frontend/*/public 是 tracked 源文件，vite 原样拷进 产物（plugin-manager/dist、static/react/neko-chat），而这两处都 gitignore， 于是一个 public/中文.png 既不在扫描面内又实打实进包。两个 public 加进 BUNDLED_ROOTS。 2) config/ 与 data/ 扫过头。构建实际只打 config 的 5 个具名文件加 characters/changelog/surveys 三个目录、data 的 browser_use_prompts/ tiktoken_cache/embedding_models 三个子树（--include-package=config 只编 Python 模块，不带数据）。原来扫整个根目录会因为 config/prompts/说明.md 之类把无关 PR 打红。改为逐条列出真正进包的条目。 3) 插件暂存规则没应用。prepare_nuitka_plugins.py 会按各插件的 [tool.neko.build] 过滤后才装进包，所以 plugins/

/tests/中文.json、 .db/.log 这些根本不进 Contents/MacOS，却会被这个 lint 判红。补上镜像版 过滤器（tomllib + fnmatch）。 之所以是镜像而非 import：真规则挂在 pydantic 下，而 analyze job 用的是 不装依赖的裸解释器。镜像用测试钉住——但两个漂移方向不对称，这一点第一版 写反了：镜像里多列一项（真实暂存会保留的）会让我们停止扫描一个真进包的 文件，那是洞；少列一项只是残留误报。所以镜像必须是真规则的子集，测试也 只对"洞"那个方向硬失败。第一版把 .vscode/.idea 写进了镜像（那是 #2871 才加进真规则的），测试当场把这个洞抓了出来，已移除。 4) ZIP 反斜杠分隔符。Windows 上打的 zip 成员形如 `中文目录\plain.png`， PurePosixPath 不认反斜杠，整串被当成 basename 而误判；真正的解包器 (_safe_relative_path) 会把 \ 换成 /，落地是"CJK 目录 + ASCII 文件名"， 按本检查的既定策略是允许的。改为同样先归一化。 另把 tomllib 的 import 加了保护：拿不到就是不知道各插件规则，等于扫得更多 （误报方向），不会漏。 339 个 offender 数量不变，基线无需重算。新增 5 条测试（镜像与真规则对拍、 被排除的插件路径不报、Windows zip 分隔符、vite public 扫到而 src 不扫、 config/data 只扫具名条目），25 条全绿，ruff 干净。 Co-Authored-By: Claude Opus 5 --- scripts/check_no_nonascii_asset_names.py | 138 +++++++++++++- .../test_check_no_nonascii_asset_names.py | 172 +++++++++++++++++- 2 files changed, 305 insertions(+), 5 deletions(-) diff --git a/scripts/check_no_nonascii_asset_names.py b/scripts/check_no_nonascii_asset_names.py index 3b3524d2c5..9878cd2c91 100644 --- a/scripts/check_no_nonascii_asset_names.py +++ b/scripts/check_no_nonascii_asset_names.py @@ -137,6 +137,7 @@ from __future__ import annotations import argparse +from fnmatch import fnmatchcase import json import os import subprocess @@ -145,6 +146,13 @@ import zipfile from pathlib import Path, PurePosixPath +try: # 3.11+ stdlib; the repo pins ==3.11.*, this is belt-and-braces + import tomllib +except ModuleNotFoundError: # pragma: no cover - only on an older interpreter + # Without it we simply do not know each plugin's [tool.neko.build] rules, + # which means scanning more than ships — false positives, never a hole. + tomllib = None + REPO_ROOT = Path(__file__).resolve().parent.parent BASELINE_PATH = REPO_ROOT / "scripts" / "nonascii_asset_baseline.txt" @@ -156,10 +164,24 @@ # new payload directory is added there. BUNDLED_ROOTS: tuple[str, ...] = ( "static", - "config", "templates", "assets", - "data", + # config/ and data/ ship a named subset, not the whole root: + # --include-package=config compiles Python modules but copies no data, so + # everything that actually lands is listed explicitly in the build scripts. + # Scanning the roots wholesale would fail a PR over e.g. config/prompts/*.md, + # which never reaches the payload. + "config/__init__.py", + "config/api_providers.json", + "config/characters.json", + "config/core_config.json", + "config/user_preferences.json", + "config/characters", + "config/changelog", + "config/surveys", + "data/browser_use_prompts", + "data/tiktoken_cache", + "data/embedding_models", # Not all of docs/ — the build packs exactly one subtree # (--include-data-dir=docs/zh-CN/guide). Scanning the whole root would fail # documentation PRs over files that never enter the payload. @@ -170,6 +192,12 @@ # static/ — so scanning all of frontend/ would fail a PR over a file that # cannot possibly break signing. "frontend/plugin-manager/dist", + # Vite copies public/ verbatim into the build output, keeping the filename. + # Those outputs are gitignored (plugin-manager/dist, static/react/neko-chat), + # so without these two roots a tracked public/ asset is invisible here yet + # still lands in the payload. + "frontend/plugin-manager/public", + "frontend/react-neko-chat/public", "plugin/plugins", # Voice-turn + speaker models, both --include-data-dir'd. The .onnx weights # are downloaded at build time and gitignored, so the git listing cannot see @@ -233,6 +261,100 @@ def _under_bundled_root(rel_posix: str) -> bool: ) +# Mirror of the staging filter in scripts/prepare_nuitka_plugins.py, which runs +# each plugin's ``[tool.neko.build]`` rules (plus hard defaults) before the +# payload is installed into the bundle. Without it this check rejects files that +# never reach Contents/MacOS — a plugin's own tests/, its .db/.log runtime +# leftovers, editor directories. +# +# It is a mirror rather than an import on purpose: the real rules live behind +# pydantic (plugin/neko_plugin_cli/core/build_rules.py) and the analyze job runs +# these scripts on a bare interpreter with no dependencies installed. +# +# The two directions of drift are not symmetric, so keep this list a SUBSET of +# the real one. Listing something the real staging keeps means we stop scanning +# a file that does ship — a hole. Missing something it drops only leaves a +# false positive. tests/unit/test_check_no_nonascii_asset_names.py imports the +# real ``should_skip_path`` and fails on the first direction. +_PLUGIN_SKIP_DIR_NAMES = frozenset( + {"__pycache__", ".github", ".pytest_cache", ".venv", ".git"} +) +_PLUGIN_SKIP_ROOT_DIR_NAMES = frozenset({"dist", "build"}) +_PLUGIN_SKIP_FILE_NAMES = frozenset({".DS_Store"}) +# .pyc/.pyo come from the build rules; .db/.log are stripped unconditionally by +# _remove_private_runtime_artifacts after staging. +_PLUGIN_SKIP_SUFFIXES = frozenset({".pyc", ".pyo", ".db", ".log"}) + +PLUGINS_ROOT = "plugin/plugins" + + +def _match_build_pattern(path_str: str, pattern: str) -> bool: + if fnmatchcase(path_str, pattern): + return True + return "/" not in pattern and fnmatchcase(PurePosixPath(path_str).name, pattern) + + +def _plugin_rules(repo_root: Path, plugin_dir: str) -> dict[str, list[str]]: + pyproject = repo_root / PLUGINS_ROOT / plugin_dir / "pyproject.toml" + if tomllib is None or not pyproject.is_file(): + return {} + try: + table = tomllib.loads(pyproject.read_text(encoding="utf-8")) + except (OSError, tomllib.TOMLDecodeError): + # A broken pyproject is the plugin CLI's problem to report; here it just + # means "no extra rules", which only ever widens what we scan. + return {} + build = table.get("tool", {}).get("neko", {}).get("build", {}) + if not isinstance(build, dict): + return {} + return { + key: [item for item in build.get(key, []) if isinstance(item, str)] + for key in ("exclude", "exclude_dirs", "exclude_files") + } + + +def _plugin_stage_filter(repo_root: Path): + """Return ``keep(path)`` — False for repo paths the plugin stage drops.""" + cache: dict[str, dict[str, list[str]]] = {} + + def keep(path: str) -> bool: + prefix = PLUGINS_ROOT + "/" + if not path.startswith(prefix): + return True + parts = PurePosixPath(path[len(prefix):]).parts + if len(parts) < 2: + return True # a loose file directly under plugin/plugins/ + plugin_dir, relative = parts[0], PurePosixPath(*parts[1:]) + dir_parts = relative.parts[:-1] + if dir_parts and dir_parts[0] in _PLUGIN_SKIP_ROOT_DIR_NAMES: + return False + if any(part in _PLUGIN_SKIP_DIR_NAMES for part in dir_parts): + return False + if relative.name in _PLUGIN_SKIP_FILE_NAMES: + return False + if relative.suffix.lower() in _PLUGIN_SKIP_SUFFIXES: + return False + + rules = cache.setdefault(plugin_dir, _plugin_rules(repo_root, plugin_dir)) + if not rules: + return True + path_str = relative.as_posix() + if any(_match_build_pattern(path_str, p) for p in rules.get("exclude", [])): + return False + for index in range(len(dir_parts)): + candidate = "/".join(dir_parts[: index + 1]) + if any( + _match_build_pattern(candidate, p) for p in rules.get("exclude_dirs", []) + ): + return False + exclude_files = rules.get("exclude_files", []) + if relative.name in exclude_files: + return False + return not any(_match_build_pattern(path_str, p) for p in exclude_files) + + return keep + + def _git_listed_offenders(repo_root: Path) -> set[str]: """Non-ASCII git-visible paths under the bundled roots. @@ -257,11 +379,13 @@ def _git_listed_offenders(repo_root: Path) -> set[str]: print(f"error: cannot list git files: {exc}", file=sys.stderr) raise SystemExit(2) from exc + staged = _plugin_stage_filter(repo_root) return { path for path in raw.split("\0") if path and _under_bundled_root(path) + and staged(path) and not _is_ascii(PurePosixPath(path).name) } @@ -277,9 +401,15 @@ def _archive_offenders(repo_root: Path) -> dict[str, str]: def _record(members: list[str], dest_prefix: str, archive_rel: str) -> None: for member in members: - if _is_ascii(PurePosixPath(member).name): + # A ZIP written on Windows can carry backslash separators; the + # unpacker normalizes them (`_safe_relative_path`), so a member like + # `中文目录\plain.png` really lands as an ASCII file inside a CJK + # directory — allowed. Without this the whole string reads as one + # basename and the check rejects a file that signs fine. + normalized = member.replace("\\", "/") + if _is_ascii(PurePosixPath(normalized).name): continue - offenders[f"{dest_prefix}/{member}"] = archive_rel + offenders[f"{dest_prefix}/{normalized}"] = archive_rel for source_dir, dest_prefix in TAR_ARCHIVE_DESTS: directory = repo_root / source_dir diff --git a/tests/unit/test_check_no_nonascii_asset_names.py b/tests/unit/test_check_no_nonascii_asset_names.py index 166c0f0d72..3cc087cd55 100644 --- a/tests/unit/test_check_no_nonascii_asset_names.py +++ b/tests/unit/test_check_no_nonascii_asset_names.py @@ -23,7 +23,7 @@ import sys import tarfile import zipfile -from pathlib import Path +from pathlib import Path, PurePosixPath import pytest @@ -517,3 +517,173 @@ def test_only_the_packaged_frontend_output_is_scanned(tmp_path: Path) -> None: offenders, _ = module.collect_offenders(tmp_path) assert offenders == {f"frontend/plugin-manager/dist/assets/{CJK_NAME}.js"} + + +@pytest.mark.unit +def test_plugin_stage_filter_matches_the_real_build_rules(tmp_path: Path) -> None: + """Pin the mirrored staging filter against the implementation it copies. + + The checker cannot import plugin.neko_plugin_cli.core.build_rules (pydantic; + the analyze job runs on a bare interpreter), so it reimplements the file-side + of should_skip_path. This test imports the real one and demands the same + verdict, so the copy cannot drift unnoticed. + """ + from plugin.neko_plugin_cli.core.build_rules import load_build_rules, should_skip_path + + module = _load_script_module() + plugin_dir = tmp_path / "plugin" / "plugins" / "demo" + plugin_dir.mkdir(parents=True) + (plugin_dir / "pyproject.toml").write_text( + "\n".join( + [ + "[tool.neko.build]", + 'exclude = ["*.tmp", "secrets/*"]', + 'exclude_dirs = ["tests", "local_logs"]', + 'exclude_files = ["README.md", "*.bak"]', + ] + ), + encoding="utf-8", + ) + rules = load_build_rules( + {"tool": {"neko": {"build": { + "exclude": ["*.tmp", "secrets/*"], + "exclude_dirs": ["tests", "local_logs"], + "exclude_files": ["README.md", "*.bak"], + }}}} + ) + + keep = module._plugin_stage_filter(tmp_path) + relatives = [ + "runtime.py", + "README.md", + "notes.bak", + "scratch.tmp", + "store.db", + "runtime.log", + "cached.pyc", + ".DS_Store", + "tests/test_runtime.py", + "tests/nested/deep.json", + "local_logs/private.txt", + "secrets/token.json", + "dist/bundle.js", + "build/out.js", + "__pycache__/mod.pyc", + ".github/workflows/ci.yml", + ".vscode/settings.json", + "data layer/worker.py", + "assets/nested/ok.png", + ] + # The two directions are not symmetric. Dropping a path the real staging + # keeps means the check stops scanning a file that ships — a hole, and the + # only direction worth failing on. Keeping one it drops merely leaves a + # false positive, so it is reported but tolerated. + for relative in relatives: + mirrored_keep = keep(f"plugin/plugins/demo/{relative}") + if PurePosixPath(relative).suffix.lower() in {".db", ".log"}: + # Stripped after staging by _remove_private_runtime_artifacts rather + # than by the rules, so the real should_skip_path says nothing here. + assert not mirrored_keep, relative + continue + real_keeps = not should_skip_path(Path(relative), is_dir=False, rules=rules) + if real_keeps: + assert mirrored_keep, f"mirror drops a staged path: {relative}" + + # And it must actually drop the families it exists for. + for dropped in ( + "tests/test_runtime.py", + "local_logs/private.txt", + "secrets/token.json", + "scratch.tmp", + "README.md", + "notes.bak", + "dist/bundle.js", + "__pycache__/mod.pyc", + "store.db", + "runtime.log", + ): + assert not keep(f"plugin/plugins/demo/{dropped}"), dropped + + +@pytest.mark.unit +def test_excluded_plugin_paths_are_not_reported(tmp_path: Path) -> None: + """A CJK name under a path the stage drops must not fail the build.""" + module = _load_script_module() + _init_repo(tmp_path) + plugin_dir = tmp_path / "plugin" / "plugins" / "demo" + (plugin_dir / "tests").mkdir(parents=True) + (plugin_dir / "pyproject.toml").write_text( + '[tool.neko.build]\nexclude_dirs = ["tests"]\n', encoding="utf-8" + ) + (plugin_dir / "tests" / f"{CJK_NAME}.json").write_bytes(b"x") + (plugin_dir / f"{CJK_NAME}.db").write_bytes(b"x") + (plugin_dir / "__pycache__").mkdir() + (plugin_dir / "__pycache__" / f"{CJK_NAME}.pyc").write_bytes(b"x") + shipped = plugin_dir / "assets" + shipped.mkdir() + (shipped / f"{CJK_NAME}.png").write_bytes(b"x") + + offenders, _ = module.collect_offenders(tmp_path) + assert offenders == {f"plugin/plugins/demo/assets/{CJK_NAME}.png"} + + +@pytest.mark.unit +def test_windows_zip_separators_are_normalized(tmp_path: Path) -> None: + """`中文目录\\plain.png` is an ASCII file in a CJK folder — allowed.""" + module = _load_script_module() + _init_repo(tmp_path) + packs = tmp_path / "frontend" / "pngtuber-packs" + packs.mkdir(parents=True) + with zipfile.ZipFile(packs / "tuber.zip", "w") as archive: + archive.writestr("中文目录\\plain.png", "x") + archive.writestr(f"层\\{CJK_NAME}", "x") + (packs / "manifest.json").write_text( + json.dumps({"models": [{"folder": "tuber", "archive": "tuber.zip"}]}), + encoding="utf-8", + ) + + offenders, _ = module.collect_offenders(tmp_path) + assert offenders == {f"static/pngtuber/tuber/层/{CJK_NAME}"} + + +@pytest.mark.unit +def test_vite_public_assets_are_scanned(tmp_path: Path) -> None: + """public/ is copied verbatim into gitignored output, so scan the source.""" + module = _load_script_module() + _init_repo(tmp_path) + for rel in ("frontend/plugin-manager/public", "frontend/react-neko-chat/public"): + (tmp_path / rel).mkdir(parents=True) + (tmp_path / rel / f"{CJK_NAME}.png").write_bytes(b"x") + unpackaged = tmp_path / "frontend" / "react-neko-chat" / "src" + unpackaged.mkdir(parents=True) + (unpackaged / f"{CJK_NAME}.tsx").write_bytes(b"x") + + offenders, _ = module.collect_offenders(tmp_path) + assert offenders == { + f"frontend/plugin-manager/public/{CJK_NAME}.png", + f"frontend/react-neko-chat/public/{CJK_NAME}.png", + } + + +@pytest.mark.unit +def test_only_packaged_config_and_data_entries_are_scanned(tmp_path: Path) -> None: + """config/ and data/ ship a named subset, not the whole root.""" + module = _load_script_module() + _init_repo(tmp_path) + for rel in ( + f"config/characters/{CJK_NAME}.json", + f"config/changelog/{CJK_NAME}.md", + f"data/browser_use_prompts/{CJK_NAME}.md", + f"config/prompts/{CJK_NAME}.md", + f"data/notes/{CJK_NAME}.md", + ): + path = tmp_path / rel + path.parent.mkdir(parents=True, exist_ok=True) + path.write_bytes(b"x") + + offenders, _ = module.collect_offenders(tmp_path) + assert offenders == { + f"config/characters/{CJK_NAME}.json", + f"config/changelog/{CJK_NAME}.md", + f"data/browser_use_prompts/{CJK_NAME}.md", + } From 606d77d074d520cc5873adcf621a2d90a4ac38c5 Mon Sep 17 00:00:00 2001 From: "H.W." Date: Fri, 14 Aug 2026 14:45:53 -0700 Subject: [PATCH 05/13] =?UTF-8?q?test:=20=E6=8A=8A=E6=B5=8B=E8=AF=95=20doc?= =?UTF-8?q?string=20=E6=94=B9=E5=9B=9E=E8=8B=B1=E6=96=87=EF=BC=88DOCSTRING?= =?UTF-8?q?=5FCJK=EF=BC=89?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Prompts & i18n 的 check_docstring_no_cjk 判红：我在 test_windows_zip_separators_are_normalized 的 docstring 里直接嵌了 CJK 样例路径。这里的 CJK 属于"内容本身受测"，用 noqa 也说得过去，但把它换成 英文描述同样说得清，就不欠一条豁免了——fixture 里那个 CJK 成员名照旧。 同 job 其余四项（prompt_hygiene / llm_budget / i18n_sync / prompt_zh_tw） 本地跑过均为 0。 Co-Authored-By: Claude Opus 5 --- tests/unit/test_check_no_nonascii_asset_names.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/tests/unit/test_check_no_nonascii_asset_names.py b/tests/unit/test_check_no_nonascii_asset_names.py index 3cc087cd55..0fcb3043fd 100644 --- a/tests/unit/test_check_no_nonascii_asset_names.py +++ b/tests/unit/test_check_no_nonascii_asset_names.py @@ -629,7 +629,11 @@ def test_excluded_plugin_paths_are_not_reported(tmp_path: Path) -> None: @pytest.mark.unit def test_windows_zip_separators_are_normalized(tmp_path: Path) -> None: - """`中文目录\\plain.png` is an ASCII file in a CJK folder — allowed.""" + """A CJK directory holding an ASCII basename is allowed; only the name counts. + + The fixture writes that member with a Windows backslash separator, which is + what used to make the whole string read as a single non-ASCII basename. + """ module = _load_script_module() _init_repo(tmp_path) packs = tmp_path / "frontend" / "pngtuber-packs" From babe2a9f67bed9fe4271c79b606f9e400e023c39 Mon Sep 17 00:00:00 2001 From: "H.W." Date: Fri, 14 Aug 2026 15:01:02 -0700 Subject: [PATCH 06/13] =?UTF-8?q?fix(ci):=20=E8=A1=A5=E4=B8=89=E6=9D=A1?= =?UTF-8?q?=E6=BC=8F=E6=89=AB=EF=BC=8C=E5=85=B6=E4=B8=AD=E4=B8=80=E6=9D=A1?= =?UTF-8?q?=E6=98=AF=E9=95=9C=E5=83=8F=E5=A4=A7=E5=B0=8F=E5=86=99=E6=8A=98?= =?UTF-8?q?=E5=8F=A0=E9=80=A0=E6=88=90=E7=9A=84=E6=B4=9E?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 1) .PYC / .PYO 大小写（洞）。真规则用 Path.suffix 与小写 .pyc/.pyo 做 区分大小写比较，而 _remove_private_runtime_artifacts 只对 .db/.log 做 lower()。所以 x.PYC 是真会被暂存、真会进包的；镜像里统一 lower() 就把它 从扫描面里剔掉了——正是我上一轮标记为"洞"的那个漂移方向，自己又踩了一次。 拆成 EXACT(.pyc/.pyo) 与 FOLDED(.db/.log) 两组，对拍矩阵补上 CACHED.PYC 与 cached.PYO。 2) vite 导入型源资源漏扫。imported asset 的输出名保留源 basename （react-neko-chat 显式配了 assets/[name]-[hash][extname]，plugin-manager 走 vite 默认值同形），而 dist 是 gitignore 的、CI 又不带 --include-untracked，于是 src/assets 下的非 ASCII 名能一路进包而检查全绿。 两个 src/assets 加进扫描面。代码模块不扫（会被打成 chunk）；动态 import 的模块名仍可能进 chunk 名，这点残留只有构建后 --include-untracked 能看到， 注释里写明了。 3) browser_use 提示词模板名漏扫。data/ 是 gitignore 的，analyze job 又不装 依赖，所以这个目录在 CI 里恒为空，而构建时会把已安装包里的 system_prompts/*.md 拷进去再打包。检查器结构上看不到，闸门只能设在生成 现场：build-desktop.yml 拷贝那步现在先校验文件名，非 ASCII 直接 SystemExit 并说明为什么会打断签名。 339 个 offender 数量不变。新增 2 条测试（大写字节码仍被扫、vite src/assets 扫而 src/components 不扫），27 条全绿；docstring-CJK、ruff、检查器本身均通过。 Co-Authored-By: Claude Opus 5 --- .github/workflows/build-desktop.yml | 11 ++++++ scripts/check_no_nonascii_asset_names.py | 23 +++++++++-- .../test_check_no_nonascii_asset_names.py | 39 +++++++++++++++++++ 3 files changed, 69 insertions(+), 4 deletions(-) diff --git a/.github/workflows/build-desktop.yml b/.github/workflows/build-desktop.yml index 647ead704c..41c7188dcf 100644 --- a/.github/workflows/build-desktop.yml +++ b/.github/workflows/build-desktop.yml @@ -235,6 +235,17 @@ jobs: src = Path(sp.__file__).parent dst = Path('data/browser_use_prompts') dst.mkdir(parents=True, exist_ok=True) + bad = [md.name for md in src.glob('*.md') if not md.name.isascii()] + if bad: + # Non-ASCII basenames under Contents/MacOS/ make codesign emit a + # hex-literal identifier and the whole bundle then fails to seal. + # scripts/check_no_nonascii_asset_names.py cannot see this one: + # data/ is gitignored and the analyze job installs no dependencies, + # so the names only exist here, right after generation. + raise SystemExit( + 'non-ASCII browser_use prompt template name(s), which break ' + f'macOS signing: {bad}' + ) for md in src.glob('*.md'): shutil.copy2(md, dst / md.name) print(f'Copied {md.name}') diff --git a/scripts/check_no_nonascii_asset_names.py b/scripts/check_no_nonascii_asset_names.py index 9878cd2c91..d467e71fd3 100644 --- a/scripts/check_no_nonascii_asset_names.py +++ b/scripts/check_no_nonascii_asset_names.py @@ -198,6 +198,15 @@ # still lands in the payload. "frontend/plugin-manager/public", "frontend/react-neko-chat/public", + # Imported assets keep their source basename in the output + # (`assets/[name]-[hash][extname]`, explicit in react-neko-chat's config and + # the Vite default elsewhere), so a non-ASCII name under src/assets/ ships + # as `-.png`. + # Code modules are not scanned: they are bundled into chunks. A dynamically + # imported module can still donate its basename to a chunk name — that + # residue is only visible to the post-build --include-untracked sweep. + "frontend/plugin-manager/src/assets", + "frontend/react-neko-chat/src/assets", "plugin/plugins", # Voice-turn + speaker models, both --include-data-dir'd. The .onnx weights # are downloaded at build time and gitignored, so the git listing cannot see @@ -281,9 +290,13 @@ def _under_bundled_root(rel_posix: str) -> bool: ) _PLUGIN_SKIP_ROOT_DIR_NAMES = frozenset({"dist", "build"}) _PLUGIN_SKIP_FILE_NAMES = frozenset({".DS_Store"}) -# .pyc/.pyo come from the build rules; .db/.log are stripped unconditionally by -# _remove_private_runtime_artifacts after staging. -_PLUGIN_SKIP_SUFFIXES = frozenset({".pyc", ".pyo", ".db", ".log"}) +# Two different comparisons upstream, and the difference matters: the build +# rules test `Path.suffix` case-SENSITIVELY against lowercase .pyc/.pyo, while +# _remove_private_runtime_artifacts lowercases before testing .db/.log. So a +# file named `x.PYC` really is staged and shipped — folding case here would +# drop it from the scan and hide it. +_PLUGIN_SKIP_SUFFIXES_EXACT = frozenset({".pyc", ".pyo"}) +_PLUGIN_SKIP_SUFFIXES_FOLDED = frozenset({".db", ".log"}) PLUGINS_ROOT = "plugin/plugins" @@ -332,7 +345,9 @@ def keep(path: str) -> bool: return False if relative.name in _PLUGIN_SKIP_FILE_NAMES: return False - if relative.suffix.lower() in _PLUGIN_SKIP_SUFFIXES: + if relative.suffix in _PLUGIN_SKIP_SUFFIXES_EXACT: + return False + if relative.suffix.lower() in _PLUGIN_SKIP_SUFFIXES_FOLDED: return False rules = cache.setdefault(plugin_dir, _plugin_rules(repo_root, plugin_dir)) diff --git a/tests/unit/test_check_no_nonascii_asset_names.py b/tests/unit/test_check_no_nonascii_asset_names.py index 0fcb3043fd..f49961fac8 100644 --- a/tests/unit/test_check_no_nonascii_asset_names.py +++ b/tests/unit/test_check_no_nonascii_asset_names.py @@ -561,6 +561,8 @@ def test_plugin_stage_filter_matches_the_real_build_rules(tmp_path: Path) -> Non "store.db", "runtime.log", "cached.pyc", + "CACHED.PYC", + "cached.PYO", ".DS_Store", "tests/test_runtime.py", "tests/nested/deep.json", @@ -691,3 +693,40 @@ def test_only_packaged_config_and_data_entries_are_scanned(tmp_path: Path) -> No f"config/changelog/{CJK_NAME}.md", f"data/browser_use_prompts/{CJK_NAME}.md", } + + +@pytest.mark.unit +def test_uppercase_bytecode_suffix_is_still_scanned(tmp_path: Path) -> None: + """The build rules compare suffixes case-sensitively, so `.PYC` ships. + + Folding case in the mirrored filter would drop such a file from the scan + while staging still installs it — the one drift direction that hides a + signing break rather than merely adding noise. + """ + module = _load_script_module() + _init_repo(tmp_path) + plugin_dir = tmp_path / "plugin" / "plugins" / "demo" + plugin_dir.mkdir(parents=True) + (plugin_dir / f"{CJK_NAME}.PYC").write_bytes(b"x") + (plugin_dir / f"{CJK_NAME}.pyc").write_bytes(b"x") + # .db/.log are stripped case-insensitively after staging, so both go. + (plugin_dir / f"{CJK_NAME}.DB").write_bytes(b"x") + + offenders, _ = module.collect_offenders(tmp_path) + assert offenders == {f"plugin/plugins/demo/{CJK_NAME}.PYC"} + + +@pytest.mark.unit +def test_vite_imported_source_assets_are_scanned(tmp_path: Path) -> None: + """Imported assets keep their basename in the output; code does not.""" + module = _load_script_module() + _init_repo(tmp_path) + assets = tmp_path / "frontend" / "plugin-manager" / "src" / "assets" + assets.mkdir(parents=True) + (assets / f"{CJK_NAME}.png").write_bytes(b"x") + code = tmp_path / "frontend" / "plugin-manager" / "src" / "components" + code.mkdir(parents=True) + (code / f"{CJK_NAME}.vue").write_bytes(b"x") + + offenders, _ = module.collect_offenders(tmp_path) + assert offenders == {f"frontend/plugin-manager/src/assets/{CJK_NAME}.png"} From 8870f7511e2668aa621eff0cfcc46f966465a3c9 Mon Sep 17 00:00:00 2001 From: "H.W." Date: Fri, 14 Aug 2026 15:06:49 -0700 Subject: [PATCH 07/13] =?UTF-8?q?fix(ci):=20=E9=95=9C=E5=83=8F=E8=A1=A5?= =?UTF-8?q?=E9=BD=90=20include=20=E7=99=BD=E5=90=8D=E5=8D=95=E8=AF=AD?= =?UTF-8?q?=E4=B9=89=EF=BC=8C=E5=AF=B9=E6=8B=8D=E6=94=B9=E4=B8=BA=E5=8F=8C?= =?UTF-8?q?=E5=90=91?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 评审指出 _plugin_stage_filter 没实现 rules.include。属实：should_skip_path 把 include 当作跑完所有 exclude 之后的 allow-list，未匹配即不进暂存；镜像 少了这一段，插件一旦配 include，检查器就会扫到构建阶段丢掉的文件，让 CI 为不进包的资源变红。目前仓库里还没有插件用 include，所以是潜伏的误报， 但补上只多六行。 对拍测试也按评审的意见改成双向相等，fixture 加了 include 规则与两条不被 它匹配的路径。上一版我只对"洞"那个方向硬失败，理由是想避免与 #2871 （往真规则里加 .vscode/.idea）的合并顺序耦合；双向更强，代价是 #2871 合入后 本文件需要同步一行——测试会直接指出是哪一项，这个代价可以接受。 保留的唯一不对称是 .db/.log：它们由 _remove_private_runtime_artifacts 在 暂存之后清理，不走这套规则，should_skip_path 对其没有意见。 验证：把 include 那段摘掉，对拍测试立刻红（"mirror and build rules disagree on CACHED.PYC: mirror keeps=True, staging keeps=False"）。另加一条端到端 测试：插件配 include=["assets/*"] 时，assets 下的非 ASCII 文件照报、docs 下 的不报。28 条全绿，检查器、docstring-CJK、ruff 均通过。 Co-Authored-By: Claude Opus 5 --- scripts/check_no_nonascii_asset_names.py | 22 +++++--- .../test_check_no_nonascii_asset_names.py | 55 ++++++++++++++----- 2 files changed, 55 insertions(+), 22 deletions(-) diff --git a/scripts/check_no_nonascii_asset_names.py b/scripts/check_no_nonascii_asset_names.py index d467e71fd3..17f7072cbd 100644 --- a/scripts/check_no_nonascii_asset_names.py +++ b/scripts/check_no_nonascii_asset_names.py @@ -280,11 +280,12 @@ def _under_bundled_root(rel_posix: str) -> bool: # pydantic (plugin/neko_plugin_cli/core/build_rules.py) and the analyze job runs # these scripts on a bare interpreter with no dependencies installed. # -# The two directions of drift are not symmetric, so keep this list a SUBSET of -# the real one. Listing something the real staging keeps means we stop scanning -# a file that does ship — a hole. Missing something it drops only leaves a -# false positive. tests/unit/test_check_no_nonascii_asset_names.py imports the -# real ``should_skip_path`` and fails on the first direction. +# tests/unit/test_check_no_nonascii_asset_names.py imports the real +# ``should_skip_path`` and demands the same verdict in both directions, so this +# copy cannot drift silently. The two directions are not equally bad, which is +# worth knowing when one does slip: listing something the real staging keeps +# stops us scanning a file that ships (a hole), while missing something it drops +# only leaves a false positive. Both fail the test; only the first is dangerous. _PLUGIN_SKIP_DIR_NAMES = frozenset( {"__pycache__", ".github", ".pytest_cache", ".venv", ".git"} ) @@ -322,7 +323,7 @@ def _plugin_rules(repo_root: Path, plugin_dir: str) -> dict[str, list[str]]: return {} return { key: [item for item in build.get(key, []) if isinstance(item, str)] - for key in ("exclude", "exclude_dirs", "exclude_files") + for key in ("include", "exclude", "exclude_dirs", "exclude_files") } @@ -365,7 +366,14 @@ def keep(path: str) -> bool: exclude_files = rules.get("exclude_files", []) if relative.name in exclude_files: return False - return not any(_match_build_pattern(path_str, p) for p in exclude_files) + if any(_match_build_pattern(path_str, p) for p in exclude_files): + return False + # `include` is an allow-list applied after every exclude has run: with + # it present, anything unmatched is dropped from the stage. + include = rules.get("include", []) + if not include: + return True + return any(_match_build_pattern(path_str, p) for p in include) return keep diff --git a/tests/unit/test_check_no_nonascii_asset_names.py b/tests/unit/test_check_no_nonascii_asset_names.py index f49961fac8..ac46890fbd 100644 --- a/tests/unit/test_check_no_nonascii_asset_names.py +++ b/tests/unit/test_check_no_nonascii_asset_names.py @@ -533,10 +533,17 @@ def test_plugin_stage_filter_matches_the_real_build_rules(tmp_path: Path) -> Non module = _load_script_module() plugin_dir = tmp_path / "plugin" / "plugins" / "demo" plugin_dir.mkdir(parents=True) + build_table = { + "include": ["*.py", "assets/*", "data layer/*"], + "exclude": ["*.tmp", "secrets/*"], + "exclude_dirs": ["tests", "local_logs"], + "exclude_files": ["README.md", "*.bak"], + } (plugin_dir / "pyproject.toml").write_text( "\n".join( [ "[tool.neko.build]", + 'include = ["*.py", "assets/*", "data layer/*"]', 'exclude = ["*.tmp", "secrets/*"]', 'exclude_dirs = ["tests", "local_logs"]', 'exclude_files = ["README.md", "*.bak"]', @@ -544,13 +551,7 @@ def test_plugin_stage_filter_matches_the_real_build_rules(tmp_path: Path) -> Non ), encoding="utf-8", ) - rules = load_build_rules( - {"tool": {"neko": {"build": { - "exclude": ["*.tmp", "secrets/*"], - "exclude_dirs": ["tests", "local_logs"], - "exclude_files": ["README.md", "*.bak"], - }}}} - ) + rules = load_build_rules({"tool": {"neko": {"build": build_table}}}) keep = module._plugin_stage_filter(tmp_path) relatives = [ @@ -575,21 +576,26 @@ def test_plugin_stage_filter_matches_the_real_build_rules(tmp_path: Path) -> Non ".vscode/settings.json", "data layer/worker.py", "assets/nested/ok.png", + # Not matched by any `include` pattern -> dropped by the allow-list. + "docs/manual.md", + "styles/theme.css", ] - # The two directions are not symmetric. Dropping a path the real staging - # keeps means the check stops scanning a file that ships — a hole, and the - # only direction worth failing on. Keeping one it drops merely leaves a - # false positive, so it is reported but tolerated. + # Demand the same verdict in both directions. One direction is a hole (the + # mirror drops a path that really ships, so the check stops looking at it), + # the other only a false positive — but a mismatch either way means the copy + # has drifted, and the fix is the same one line. The sole documented + # asymmetry is .db/.log, which staging strips afterwards rather than through + # these rules, so should_skip_path has no opinion on them. for relative in relatives: mirrored_keep = keep(f"plugin/plugins/demo/{relative}") if PurePosixPath(relative).suffix.lower() in {".db", ".log"}: - # Stripped after staging by _remove_private_runtime_artifacts rather - # than by the rules, so the real should_skip_path says nothing here. assert not mirrored_keep, relative continue real_keeps = not should_skip_path(Path(relative), is_dir=False, rules=rules) - if real_keeps: - assert mirrored_keep, f"mirror drops a staged path: {relative}" + assert mirrored_keep == real_keeps, ( + f"mirror and build rules disagree on {relative}: " + f"mirror keeps={mirrored_keep}, staging keeps={real_keeps}" + ) # And it must actually drop the families it exists for. for dropped in ( @@ -603,6 +609,7 @@ def test_plugin_stage_filter_matches_the_real_build_rules(tmp_path: Path) -> Non "__pycache__/mod.pyc", "store.db", "runtime.log", + "docs/manual.md", ): assert not keep(f"plugin/plugins/demo/{dropped}"), dropped @@ -730,3 +737,21 @@ def test_vite_imported_source_assets_are_scanned(tmp_path: Path) -> None: offenders, _ = module.collect_offenders(tmp_path) assert offenders == {f"frontend/plugin-manager/src/assets/{CJK_NAME}.png"} + + +@pytest.mark.unit +def test_include_allow_list_keeps_unpackaged_files_out_of_the_scan(tmp_path: Path) -> None: + """With `include` set, staging drops everything unmatched — so do not report it.""" + module = _load_script_module() + _init_repo(tmp_path) + plugin_dir = tmp_path / "plugin" / "plugins" / "demo" + (plugin_dir / "assets").mkdir(parents=True) + (plugin_dir / "docs").mkdir() + (plugin_dir / "pyproject.toml").write_text( + '[tool.neko.build]\ninclude = ["assets/*"]\n', encoding="utf-8" + ) + (plugin_dir / "assets" / f"{CJK_NAME}.png").write_bytes(b"x") + (plugin_dir / "docs" / f"{CJK_NAME}.md").write_bytes(b"x") + + offenders, _ = module.collect_offenders(tmp_path) + assert offenders == {f"plugin/plugins/demo/assets/{CJK_NAME}.png"} From c52bed644d1d5c97579120d189ee6e6769df17fb Mon Sep 17 00:00:00 2001 From: "H.W." Date: Fri, 14 Aug 2026 15:17:29 -0700 Subject: [PATCH 08/13] =?UTF-8?q?fix(ci):=20=E5=9F=BA=E7=BA=BF=E6=AF=94?= =?UTF-8?q?=E5=AF=B9=E6=94=B9=E7=94=A8=20merge-base=EF=BC=9B=E5=89=8D?= =?UTF-8?q?=E7=AB=AF=E6=89=AB=E6=8F=8F=E9=9D=A2=E8=BF=98=E5=8E=9F=E4=B8=BA?= =?UTF-8?q?=E6=95=B4=E6=A3=B5=20src?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 1) --base 比的是 origin/main 的 tip，不是 merge-base——注释、工作流说明和 测试名却都写着 merge-base。后果是真的：main 在分支切出后删掉几条陈旧基线 条目，分支里仍带着它们，tip 比对就会把这些报成"新增"，让别人的清理把你的 PR 打红。改为先 git merge-base，从那个提交读基线（拿不到再退回 base_ref）。 本仓库当前 tip 与 merge-base 就已经不是同一个提交，不是理论问题。 2) 前端扫描面从 dist + src/assets 还原为 dist + 整棵 src。这推翻了我前两轮 按评审意见做的两次收窄，理由是那两次都建立在一个错误模型上：以为"源码不 进 payload"。实际上 vite 是按 import 决定输出名的——超过内联阈值的导入资源 在任何目录下都会以 assets/[name]-[hash][extname] 落地，动态 import 的模块 还会把 basename 交给 chunk 名。两种产物都 gitignore，CI 又不带 --include-untracked，所以源码树是普通 CI 唯一看得见这些名字的地方。 代价是可能为"其实不会贡献文件名"的源文件误报一次。这个取舍是明确的： 前端源码用非 ASCII 文件名本来就罕见，改名成本一次；漏一个的成本是整个 mac 发布。src 之外的工程文件（配置、锁文件）仍不扫。 验证：把 merge-base 那段摘掉，新测试立刻红。29 条全绿；检查器、docstring-CJK、 ruff 均通过。 Co-Authored-By: Claude Opus 5 --- scripts/check_no_nonascii_asset_names.py | 33 +++++-- .../test_check_no_nonascii_asset_names.py | 94 ++++++++++++++----- 2 files changed, 96 insertions(+), 31 deletions(-) diff --git a/scripts/check_no_nonascii_asset_names.py b/scripts/check_no_nonascii_asset_names.py index 17f7072cbd..12b0851a83 100644 --- a/scripts/check_no_nonascii_asset_names.py +++ b/scripts/check_no_nonascii_asset_names.py @@ -198,15 +198,15 @@ # still lands in the payload. "frontend/plugin-manager/public", "frontend/react-neko-chat/public", - # Imported assets keep their source basename in the output - # (`assets/[name]-[hash][extname]`, explicit in react-neko-chat's config and - # the Vite default elsewhere), so a non-ASCII name under src/assets/ ships - # as `-.png`. - # Code modules are not scanned: they are bundled into chunks. A dynamically - # imported module can still donate its basename to a chunk name — that - # residue is only visible to the post-build --include-untracked sweep. - "frontend/plugin-manager/src/assets", - "frontend/react-neko-chat/src/assets", + # Whole source trees, not just src/assets: Vite decides by *import*, not by + # directory. An imported asset above the inline limit is emitted as + # `assets/[name]-[hash][extname]` wherever it lives, and a dynamically + # imported module donates its basename to the chunk name the same way. Both + # outputs are gitignored, so the source tree is the only place CI can see + # these names without building. An ASCII-only rule for frontend sources + # costs a rename; missing one costs the whole mac release. + "frontend/plugin-manager/src", + "frontend/react-neko-chat/src", "plugin/plugins", # Voice-turn + speaker models, both --include-data-dir'd. The .onnx weights # are downloaded at build time and gitignored, so the git listing cannot see @@ -639,9 +639,22 @@ def _baseline_growth(repo_root: Path, base_ref: str) -> list[str]: ) raise SystemExit(2) + # The merge base, not the tip of base_ref. If main drops grandfathered + # entries after this branch was cut, the branch still carries them and a + # tip comparison would report those as newly added — a red build for + # somebody else's cleanup. + merge_base = subprocess.run( + ["git", "merge-base", base_ref, "HEAD"], + cwd=repo_root, + capture_output=True, + ) + reference = ( + merge_base.stdout.decode().strip() if merge_base.returncode == 0 else base_ref + ) + rel = BASELINE_PATH.relative_to(repo_root).as_posix() completed = subprocess.run( - ["git", "show", f"{base_ref}:{rel}"], + ["git", "show", f"{reference}:{rel}"], cwd=repo_root, capture_output=True, ) diff --git a/tests/unit/test_check_no_nonascii_asset_names.py b/tests/unit/test_check_no_nonascii_asset_names.py index ac46890fbd..e200c37aa6 100644 --- a/tests/unit/test_check_no_nonascii_asset_names.py +++ b/tests/unit/test_check_no_nonascii_asset_names.py @@ -504,19 +504,26 @@ def test_unresolvable_base_ref_fails_instead_of_passing(tmp_path: Path) -> None: @pytest.mark.unit -def test_only_the_packaged_frontend_output_is_scanned(tmp_path: Path) -> None: - """All three build scripts pack frontend/plugin-manager/dist and nothing else.""" +def test_frontend_scan_covers_output_and_sources_but_not_tooling(tmp_path: Path) -> None: + """dist/ is what ships; src/ is where the emitted names come from. + + Files outside both — configs, lockfiles, node tooling — never contribute a + basename to the payload, so they stay out of scope. + """ module = _load_script_module() _init_repo(tmp_path) - source = tmp_path / "frontend" / "plugin-manager" / "src" - source.mkdir(parents=True) - (source / f"{CJK_NAME}.vue").write_bytes(b"x") - packaged = tmp_path / "frontend" / "plugin-manager" / "dist" / "assets" - packaged.mkdir(parents=True) - (packaged / f"{CJK_NAME}.js").write_bytes(b"x") + root = tmp_path / "frontend" / "plugin-manager" + (root / "src").mkdir(parents=True) + (root / "dist" / "assets").mkdir(parents=True) + (root / "src" / f"{CJK_NAME}.vue").write_bytes(b"x") + (root / "dist" / "assets" / f"{CJK_NAME}.js").write_bytes(b"x") + (root / f"{CJK_NAME}.config.ts").write_bytes(b"x") offenders, _ = module.collect_offenders(tmp_path) - assert offenders == {f"frontend/plugin-manager/dist/assets/{CJK_NAME}.js"} + assert offenders == { + f"frontend/plugin-manager/src/{CJK_NAME}.vue", + f"frontend/plugin-manager/dist/assets/{CJK_NAME}.js", + } @pytest.mark.unit @@ -667,9 +674,7 @@ def test_vite_public_assets_are_scanned(tmp_path: Path) -> None: for rel in ("frontend/plugin-manager/public", "frontend/react-neko-chat/public"): (tmp_path / rel).mkdir(parents=True) (tmp_path / rel / f"{CJK_NAME}.png").write_bytes(b"x") - unpackaged = tmp_path / "frontend" / "react-neko-chat" / "src" - unpackaged.mkdir(parents=True) - (unpackaged / f"{CJK_NAME}.tsx").write_bytes(b"x") + (tmp_path / "frontend" / "react-neko-chat" / f"{CJK_NAME}.config.ts").write_bytes(b"x") offenders, _ = module.collect_offenders(tmp_path) assert offenders == { @@ -724,19 +729,31 @@ def test_uppercase_bytecode_suffix_is_still_scanned(tmp_path: Path) -> None: @pytest.mark.unit -def test_vite_imported_source_assets_are_scanned(tmp_path: Path) -> None: - """Imported assets keep their basename in the output; code does not.""" +def test_vite_source_basenames_are_scanned_anywhere_under_src(tmp_path: Path) -> None: + """Vite emits by import, not by directory, so the whole source tree counts. + + An imported asset above the inline limit keeps its basename wherever it + lives, and a dynamically imported module donates its basename to the chunk + name. Both outputs are gitignored, so the source tree is the only place a + plain CI run can see these names. + """ module = _load_script_module() _init_repo(tmp_path) - assets = tmp_path / "frontend" / "plugin-manager" / "src" / "assets" - assets.mkdir(parents=True) - (assets / f"{CJK_NAME}.png").write_bytes(b"x") - code = tmp_path / "frontend" / "plugin-manager" / "src" / "components" - code.mkdir(parents=True) - (code / f"{CJK_NAME}.vue").write_bytes(b"x") + for rel in ("src/assets", "src/components", "src/views"): + (tmp_path / "frontend" / "plugin-manager" / rel).mkdir(parents=True) + root = tmp_path / "frontend" / "plugin-manager" + (root / "src" / "assets" / f"{CJK_NAME}.png").write_bytes(b"x") + (root / "src" / "components" / f"{CJK_NAME}.png").write_bytes(b"x") + (root / "src" / "views" / f"{CJK_NAME}.vue").write_bytes(b"x") + # Outside any source tree: build tooling, never emitted with its own name. + (root / f"{CJK_NAME}.config.ts").write_bytes(b"x") offenders, _ = module.collect_offenders(tmp_path) - assert offenders == {f"frontend/plugin-manager/src/assets/{CJK_NAME}.png"} + assert offenders == { + f"frontend/plugin-manager/src/assets/{CJK_NAME}.png", + f"frontend/plugin-manager/src/components/{CJK_NAME}.png", + f"frontend/plugin-manager/src/views/{CJK_NAME}.vue", + } @pytest.mark.unit @@ -755,3 +772,38 @@ def test_include_allow_list_keeps_unpackaged_files_out_of_the_scan(tmp_path: Pat offenders, _ = module.collect_offenders(tmp_path) assert offenders == {f"plugin/plugins/demo/assets/{CJK_NAME}.png"} + + +@pytest.mark.unit +def test_baseline_growth_is_measured_at_the_merge_base(tmp_path: Path) -> None: + """A cleanup landing on main after the branch was cut must not read as growth. + + Comparing against the tip would show entries the branch never added — they + were present when it was created and only removed on main afterwards. + """ + module = _load_script_module() + _init_repo(tmp_path) + assets = tmp_path / "static" / "audio" + assets.mkdir(parents=True) + (assets / CJK_NAME).write_bytes(b"x") + _write_baseline(tmp_path, [f"static/audio/{CJK_NAME}"]) + commit = ["-c", "user.email=t@t", "-c", "user.name=t", "commit", "-qm"] + subprocess.run(["git", "add", "-A"], cwd=tmp_path, check=True) + subprocess.run(["git", *commit, "shared base"], cwd=tmp_path, check=True) + subprocess.run(["git", "branch", "feature"], cwd=tmp_path, check=True) + + # main removes the grandfathered asset and its baseline line. + (assets / CJK_NAME).unlink() + _write_baseline(tmp_path, []) + subprocess.run(["git", "add", "-A"], cwd=tmp_path, check=True) + subprocess.run(["git", *commit, "main: shrink baseline"], cwd=tmp_path, check=True) + + # The feature branch still carries the entry it inherited. + subprocess.run(["git", "checkout", "-q", "feature"], cwd=tmp_path, check=True) + module.REPO_ROOT = tmp_path + module.BASELINE_PATH = tmp_path / "scripts" / "nonascii_asset_baseline.txt" + assert module._baseline_growth(tmp_path, "main") == [] + + # …while a line this branch really adds is still caught. + _write_baseline(tmp_path, [f"static/audio/{CJK_NAME}", "static/audio/new.mp3"]) + assert module._baseline_growth(tmp_path, "main") == ["static/audio/new.mp3"] From 8470854a794f0fb53fe84739fa62496e6e1fd736 Mon Sep 17 00:00:00 2001 From: "H.W." Date: Fri, 14 Aug 2026 15:21:21 -0700 Subject: [PATCH 09/13] =?UTF-8?q?fix(ci):=20merge-base=20=E5=8F=96?= =?UTF-8?q?=E4=B8=8D=E5=88=B0=E6=97=B6=E6=8A=A5=E9=94=99=EF=BC=8C=E4=B8=8D?= =?UTF-8?q?=E5=86=8D=E9=80=80=E5=9B=9E=20tip?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 评审指出我给 merge-base 留的 fallback 会在没有共同祖先时退回比 base_ref 的 tip——那正好把上一个 commit 刚修掉的 bug 悄悄装回去。和之前"引用不可解析就 静默放行"是同一类：环境层面的问题必须响，不能降级成看起来正常的结果。 改为输出错误并 SystemExit(2)。 会走到这条分支的是：无关历史，或者浅克隆浅到共同祖先根本没被 fetch 下来 （工作流里那步已经配了 fetch-depth: 0，正是为此）。 补了回归测试：建一个 orphan 分支与 main 无共同祖先，_baseline_growth 必须 退 2。30 条全绿。 Co-Authored-By: Claude Opus 5 --- scripts/check_no_nonascii_asset_names.py | 14 +++++++--- .../test_check_no_nonascii_asset_names.py | 27 +++++++++++++++++++ 2 files changed, 38 insertions(+), 3 deletions(-) diff --git a/scripts/check_no_nonascii_asset_names.py b/scripts/check_no_nonascii_asset_names.py index 12b0851a83..93883e9b2b 100644 --- a/scripts/check_no_nonascii_asset_names.py +++ b/scripts/check_no_nonascii_asset_names.py @@ -648,9 +648,17 @@ def _baseline_growth(repo_root: Path, base_ref: str) -> list[str]: cwd=repo_root, capture_output=True, ) - reference = ( - merge_base.stdout.decode().strip() if merge_base.returncode == 0 else base_ref - ) + if merge_base.returncode != 0: + # No common ancestor: unrelated histories, or a clone shallow enough + # that the ancestor was never fetched. Falling back to the tip would + # quietly restore the bug this function exists to avoid, so say so. + print( + f"error: no merge base between {base_ref} and HEAD " + "(unrelated histories, or the shared history was not fetched)", + file=sys.stderr, + ) + raise SystemExit(2) + reference = merge_base.stdout.decode().strip() rel = BASELINE_PATH.relative_to(repo_root).as_posix() completed = subprocess.run( diff --git a/tests/unit/test_check_no_nonascii_asset_names.py b/tests/unit/test_check_no_nonascii_asset_names.py index e200c37aa6..aec2f85061 100644 --- a/tests/unit/test_check_no_nonascii_asset_names.py +++ b/tests/unit/test_check_no_nonascii_asset_names.py @@ -807,3 +807,30 @@ def test_baseline_growth_is_measured_at_the_merge_base(tmp_path: Path) -> None: # …while a line this branch really adds is still caught. _write_baseline(tmp_path, [f"static/audio/{CJK_NAME}", "static/audio/new.mp3"]) assert module._baseline_growth(tmp_path, "main") == ["static/audio/new.mp3"] + + +@pytest.mark.unit +def test_missing_merge_base_fails_instead_of_using_the_tip(tmp_path: Path) -> None: + """Unrelated histories must not silently degrade to a tip comparison. + + That fallback would restore the very bug the merge-base lookup fixes, and + it would do it invisibly — the same failure shape as an unresolvable ref. + """ + module = _load_script_module() + _init_repo(tmp_path) + commit = ["-c", "user.email=t@t", "-c", "user.name=t", "commit", "-qm"] + _write_baseline(tmp_path, ["static/audio/base.mp3"]) + subprocess.run(["git", "add", "-A"], cwd=tmp_path, check=True) + subprocess.run(["git", *commit, "main"], cwd=tmp_path, check=True) + + # An orphan branch shares no history with main. + subprocess.run(["git", "checkout", "-q", "--orphan", "unrelated"], cwd=tmp_path, check=True) + _write_baseline(tmp_path, ["static/audio/other.mp3"]) + subprocess.run(["git", "add", "-A"], cwd=tmp_path, check=True) + subprocess.run(["git", *commit, "unrelated"], cwd=tmp_path, check=True) + + module.REPO_ROOT = tmp_path + module.BASELINE_PATH = tmp_path / "scripts" / "nonascii_asset_baseline.txt" + with pytest.raises(SystemExit) as excinfo: + module._baseline_growth(tmp_path, "main") + assert excinfo.value.code == 2 From 86b8b4d4d23d2c62c9fdc303b2efd14c9f5f690c Mon Sep 17 00:00:00 2001 From: "H.W." Date: Fri, 14 Aug 2026 15:33:10 -0700 Subject: [PATCH 10/13] =?UTF-8?q?fix(ci):=20=E8=A1=A5=E4=B8=A4=E4=B8=AA?= =?UTF-8?q?=E6=B4=9E=EF=BC=88=E6=A8=A1=E5=BC=8F=E7=A9=BA=E7=99=BD=E3=80=81?= =?UTF-8?q?playwright=20=E4=BA=A7=E7=89=A9=EF=BC=89=E4=B8=8E=E4=B8=A4?= =?UTF-8?q?=E5=A4=84=E5=AF=B9=E6=8B=8D=E7=BC=BA=E5=8F=A3?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 按上一轮定的标准（只为真缺陷改代码），四条里两条是洞、两条是把"双向对拍" 这句话兑现，都改了： 洞 1：规则模式没做空白归一。真 BuildRuleSet 会 strip 每一项并丢掉空串与 重复；镜像不做的话，` assets/* ` 这种写法在镜像里匹配不上，include 白名单 随即把该插件的所有文件都判成"不进包"，检查器就此停止扫描真会进包的文件。 按真实现补齐 strip/去空/去重。 洞 2：playwright_browsers 完全在扫描面之外。它是构建时下载的，mac 分支又把 整棵树直接拷进 Contents/MacOS/，检查器结构上看不到。和 browser_use 提示词 同一处理：闸门设在生成现场——拷贝前先校验文件名，非 ASCII 直接中止。 对拍缺口 1：镜像少了 .mypy_cache（真规则里有）。方向上只是误报，但既然已经 宣称双向对拍，就不该留着已知不一致；矩阵补 .mypy_cache/module.json。 对拍缺口 2：plugin/plugins/ 根下的松散文件走了 early return，绕过了 .db/.log 的清理判据。真流程会把它们拷进暂存再被 _remove_private_runtime_artifacts 删掉，所以报出来是误报。 验证：把 strip 和 .mypy_cache 两处退回去，对拍测试立刻红，且报的正是危险 方向（"mirror keeps=False, staging keeps=True"）。playwright 闸门用真实目录 试过：干净树退 0，塞一个中文名退 1 并指名文件。31 条全绿。 Co-Authored-By: Claude Opus 5 --- .github/workflows/build-desktop.yml | 12 +++++++++ scripts/check_no_nonascii_asset_names.py | 26 +++++++++++++++--- .../test_check_no_nonascii_asset_names.py | 27 +++++++++++++++++-- 3 files changed, 60 insertions(+), 5 deletions(-) diff --git a/.github/workflows/build-desktop.yml b/.github/workflows/build-desktop.yml index 41c7188dcf..a6c01f153f 100644 --- a/.github/workflows/build-desktop.yml +++ b/.github/workflows/build-desktop.yml @@ -701,6 +701,18 @@ jobs: # macOS: playwright_browsers was excluded from Nuitka (xattr bug with # spaces in .app paths), copy it manually into the output directory if [[ "$RUNNER_OS" == "macOS" && -d "playwright_browsers" ]]; then + # This tree lands under Contents/MacOS/, where codesign treats every + # file as nested code; a non-ASCII basename becomes a hex-literal + # identifier and the whole bundle then fails to seal. The payload is + # downloaded, so scripts/check_no_nonascii_asset_names.py structurally + # cannot see it — the only place to look is right here, before the copy. + .venv/bin/python -c " + import sys + from pathlib import Path + bad = [str(p) for p in Path('playwright_browsers').rglob('*') if not p.name.isascii()] + if bad: + sys.exit('non-ASCII name(s) in playwright_browsers, which break macOS signing: ' + repr(bad[:10])) + " cp -R playwright_browsers "$RUNTIME_DIR/playwright_browsers" echo "Copied playwright_browsers into $RUNTIME_DIR/" fi diff --git a/scripts/check_no_nonascii_asset_names.py b/scripts/check_no_nonascii_asset_names.py index 93883e9b2b..258d031812 100644 --- a/scripts/check_no_nonascii_asset_names.py +++ b/scripts/check_no_nonascii_asset_names.py @@ -287,7 +287,7 @@ def _under_bundled_root(rel_posix: str) -> bool: # stops us scanning a file that ships (a hole), while missing something it drops # only leaves a false positive. Both fail the test; only the first is dangerous. _PLUGIN_SKIP_DIR_NAMES = frozenset( - {"__pycache__", ".github", ".pytest_cache", ".venv", ".git"} + {"__pycache__", ".github", ".pytest_cache", ".mypy_cache", ".venv", ".git"} ) _PLUGIN_SKIP_ROOT_DIR_NAMES = frozenset({"dist", "build"}) _PLUGIN_SKIP_FILE_NAMES = frozenset({".DS_Store"}) @@ -321,8 +321,25 @@ def _plugin_rules(repo_root: Path, plugin_dir: str) -> dict[str, list[str]]: build = table.get("tool", {}).get("neko", {}).get("build", {}) if not isinstance(build, dict): return {} + # BuildRuleSet._normalize_pattern_list strips each entry and drops blanks + # and duplicates. Skipping that here is not cosmetic: ` assets/* ` would + # fail to match, the include allow-list would then reject everything, and + # the checker would stop scanning files that really do ship. + def _patterns(key: str) -> list[str]: + seen: set[str] = set() + out: list[str] = [] + for item in build.get(key, []): + if not isinstance(item, str): + continue + pattern = item.strip() + if not pattern or pattern in seen: + continue + seen.add(pattern) + out.append(pattern) + return out + return { - key: [item for item in build.get(key, []) if isinstance(item, str)] + key: _patterns(key) for key in ("include", "exclude", "exclude_dirs", "exclude_files") } @@ -337,7 +354,10 @@ def keep(path: str) -> bool: return True parts = PurePosixPath(path[len(prefix):]).parts if len(parts) < 2: - return True # a loose file directly under plugin/plugins/ + # A loose file directly under plugin/plugins/. No plugin rules apply, + # but _remove_private_runtime_artifacts sweeps the whole stage, so + # .db/.log still go. + return PurePosixPath(parts[-1]).suffix.lower() not in _PLUGIN_SKIP_SUFFIXES_FOLDED plugin_dir, relative = parts[0], PurePosixPath(*parts[1:]) dir_parts = relative.parts[:-1] if dir_parts and dir_parts[0] in _PLUGIN_SKIP_ROOT_DIR_NAMES: diff --git a/tests/unit/test_check_no_nonascii_asset_names.py b/tests/unit/test_check_no_nonascii_asset_names.py index aec2f85061..e2491048db 100644 --- a/tests/unit/test_check_no_nonascii_asset_names.py +++ b/tests/unit/test_check_no_nonascii_asset_names.py @@ -541,7 +541,9 @@ def test_plugin_stage_filter_matches_the_real_build_rules(tmp_path: Path) -> Non plugin_dir = tmp_path / "plugin" / "plugins" / "demo" plugin_dir.mkdir(parents=True) build_table = { - "include": ["*.py", "assets/*", "data layer/*"], + # Padded on purpose: BuildRuleSet strips entries, and a mirror that + # does not would reject everything the allow-list should have kept. + "include": ["*.py", " assets/* ", "data layer/*"], "exclude": ["*.tmp", "secrets/*"], "exclude_dirs": ["tests", "local_logs"], "exclude_files": ["README.md", "*.bak"], @@ -550,7 +552,7 @@ def test_plugin_stage_filter_matches_the_real_build_rules(tmp_path: Path) -> Non "\n".join( [ "[tool.neko.build]", - 'include = ["*.py", "assets/*", "data layer/*"]', + 'include = ["*.py", " assets/* ", "data layer/*"]', 'exclude = ["*.tmp", "secrets/*"]', 'exclude_dirs = ["tests", "local_logs"]', 'exclude_files = ["README.md", "*.bak"]', @@ -581,6 +583,7 @@ def test_plugin_stage_filter_matches_the_real_build_rules(tmp_path: Path) -> Non "__pycache__/mod.pyc", ".github/workflows/ci.yml", ".vscode/settings.json", + ".mypy_cache/module.json", "data layer/worker.py", "assets/nested/ok.png", # Not matched by any `include` pattern -> dropped by the allow-list. @@ -614,6 +617,7 @@ def test_plugin_stage_filter_matches_the_real_build_rules(tmp_path: Path) -> Non "notes.bak", "dist/bundle.js", "__pycache__/mod.pyc", + ".mypy_cache/module.json", "store.db", "runtime.log", "docs/manual.md", @@ -834,3 +838,22 @@ def test_missing_merge_base_fails_instead_of_using_the_tip(tmp_path: Path) -> No with pytest.raises(SystemExit) as excinfo: module._baseline_growth(tmp_path, "main") assert excinfo.value.code == 2 + + +@pytest.mark.unit +def test_loose_plugin_runtime_artifacts_are_not_reported(tmp_path: Path) -> None: + """`.db`/`.log` directly under plugin/plugins/ are swept after staging too. + + _remove_private_runtime_artifacts walks the whole stage, including the loose + files copied from the plugins root, so reporting one is a false failure. + """ + module = _load_script_module() + _init_repo(tmp_path) + loose = tmp_path / "plugin" / "plugins" + loose.mkdir(parents=True) + (loose / f"{CJK_NAME}.db").write_bytes(b"x") + (loose / f"{CJK_NAME}.LOG").write_bytes(b"x") + (loose / f"{CJK_NAME}.json").write_bytes(b"x") + + offenders, _ = module.collect_offenders(tmp_path) + assert offenders == {f"plugin/plugins/{CJK_NAME}.json"} From 28a04e3dcdb66e5d914b17be060bcb83648ff1d3 Mon Sep 17 00:00:00 2001 From: "H.W." Date: Fri, 14 Aug 2026 15:44:54 -0700 Subject: [PATCH 11/13] =?UTF-8?q?fix(ci):=20=E6=9E=84=E5=BB=BA=E9=97=B8?= =?UTF-8?q?=E9=97=A8=E4=B8=8D=E5=86=8D=E5=9B=A0=E7=9B=AE=E5=BD=95=E5=90=8D?= =?UTF-8?q?=E6=8A=A5=E9=94=99=EF=BC=9Bexclude=20=E8=A1=A5=E4=B8=8A?= =?UTF-8?q?=E7=9B=AE=E5=BD=95=E5=89=AA=E6=9E=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 1) playwright 闸门用 rglob('*') 连目录一起判，于是一个"CJK 目录 + 全 ASCII 文件"就能让 mac 构建在拷贝前直接失败——而这个仓库已经用真证书验证过那种 结构签得过（检查器 docstring 里写着，还有配套单测）。闸门自相矛盾且卡的是 构建，比 lint 误报更贵。改为只判文件。 2) 镜像只把通用 exclude 模式与完整文件路径比，而真实 walk 会对每个目录调 should_skip_path(is_dir=True) 并整棵剪掉，所以 exclude = ["cache"] 会丢掉 cache/** 而镜像仍在扫。补上按祖先目录逐级匹配。 对拍测试同时改了两处，因为它原来抓不到 2： - 用 _really_staged 建模真实 walk（先按祖先目录剪枝，再判文件本身）， 原来只调 should_skip_path(is_dir=False)，结构上看不见目录剪枝； - 拆成"有 include"和"无 include"两组规则跑。include 白名单会把一切未匹配项 一律判掉，掩盖掉其他判据；而本仓库现在没有任何插件用 include，无 include 才是实际生效的形态。加这一组之后，摘掉目录剪枝，测试立刻报 "disagree on cache/blob.json: mirror keeps=True, staging keeps=False"。 未采纳第三条（为 data-only 根目录镜像 Nuitka 的 default_ignored_suffixes， 使 assets/中文.py 不再上报）：前提属实，--include-data-dir 确实过滤 .py。 但那要求逐个根目录区分"data-dir 打包"与"原样拷贝"——plugin/plugins 是后者， .py 真会进包——一旦分类错，误报会变成漏报。而且那份忽略列表是 Nuitka 内部 API，随版本变化（当前值里甚至含 .cpython-311-darwin.so）。收益是消除一个 从未出现过的误报形态，代价是引入一类漏报风险，不划算。 31 条全绿；检查器、docstring-CJK、ruff 均通过。 Co-Authored-By: Claude Opus 5 --- .github/workflows/build-desktop.yml | 6 ++- scripts/check_no_nonascii_asset_names.py | 7 ++- .../test_check_no_nonascii_asset_names.py | 50 ++++++++++++++----- 3 files changed, 48 insertions(+), 15 deletions(-) diff --git a/.github/workflows/build-desktop.yml b/.github/workflows/build-desktop.yml index a6c01f153f..85a5e6b280 100644 --- a/.github/workflows/build-desktop.yml +++ b/.github/workflows/build-desktop.yml @@ -709,7 +709,11 @@ jobs: .venv/bin/python -c " import sys from pathlib import Path - bad = [str(p) for p in Path('playwright_browsers').rglob('*') if not p.name.isascii()] + # Files only: a CJK *directory* holding ASCII files seals fine + # (verified against a real Developer ID cert; see the checker's + # module docstring), and failing on it would block the build for nothing. + bad = [str(p) for p in Path('playwright_browsers').rglob('*') + if p.is_file() and not p.name.isascii()] if bad: sys.exit('non-ASCII name(s) in playwright_browsers, which break macOS signing: ' + repr(bad[:10])) " diff --git a/scripts/check_no_nonascii_asset_names.py b/scripts/check_no_nonascii_asset_names.py index 258d031812..b9f97dd15e 100644 --- a/scripts/check_no_nonascii_asset_names.py +++ b/scripts/check_no_nonascii_asset_names.py @@ -377,10 +377,15 @@ def keep(path: str) -> bool: path_str = relative.as_posix() if any(_match_build_pattern(path_str, p) for p in rules.get("exclude", [])): return False + # Both lists are also tested against every ancestor directory: the real + # walk asks should_skip_path(is_dir=True) for each directory and prunes + # the whole subtree, so `exclude = ["cache"]` drops cache/** even though + # no file path equals "cache". for index in range(len(dir_parts)): candidate = "/".join(dir_parts[: index + 1]) if any( - _match_build_pattern(candidate, p) for p in rules.get("exclude_dirs", []) + _match_build_pattern(candidate, p) + for p in rules.get("exclude_dirs", []) + rules.get("exclude", []) ): return False exclude_files = rules.get("exclude_files", []) diff --git a/tests/unit/test_check_no_nonascii_asset_names.py b/tests/unit/test_check_no_nonascii_asset_names.py index e2491048db..2992ebaed4 100644 --- a/tests/unit/test_check_no_nonascii_asset_names.py +++ b/tests/unit/test_check_no_nonascii_asset_names.py @@ -540,29 +540,40 @@ def test_plugin_stage_filter_matches_the_real_build_rules(tmp_path: Path) -> Non module = _load_script_module() plugin_dir = tmp_path / "plugin" / "plugins" / "demo" plugin_dir.mkdir(parents=True) - build_table = { + # Two rule shapes, because an `include` allow-list masks every other verdict: + # with it present, anything unmatched is dropped regardless of why. No plugin + # in this repo currently uses `include`, so the no-include case is the one + # that actually runs today — and the only one where exclude-driven directory + # pruning is observable. + base_rules = { # Padded on purpose: BuildRuleSet strips entries, and a mirror that # does not would reject everything the allow-list should have kept. - "include": ["*.py", " assets/* ", "data layer/*"], - "exclude": ["*.tmp", "secrets/*"], + # "cache" matches a directory, not any file path — the real walk prunes + # the subtree through should_skip_path(is_dir=True). + "exclude": ["*.tmp", " secrets/* ", "cache"], "exclude_dirs": ["tests", "local_logs"], "exclude_files": ["README.md", "*.bak"], } + for build_table in (base_rules, {**base_rules, "include": ["*.py", " assets/* ", "data layer/*"]}): + _assert_stage_filter_parity(plugin_dir, build_table, module, should_skip_path, load_build_rules) + + +def _assert_stage_filter_parity( + plugin_dir: Path, build_table: dict, module, should_skip_path, load_build_rules +) -> None: + def _toml_list(values: list[str]) -> str: + return "[" + ", ".join(f'"{value}"' for value in values) + "]" + (plugin_dir / "pyproject.toml").write_text( "\n".join( - [ - "[tool.neko.build]", - 'include = ["*.py", " assets/* ", "data layer/*"]', - 'exclude = ["*.tmp", "secrets/*"]', - 'exclude_dirs = ["tests", "local_logs"]', - 'exclude_files = ["README.md", "*.bak"]', - ] + ["[tool.neko.build]"] + + [f"{key} = {_toml_list(value)}" for key, value in build_table.items()] ), encoding="utf-8", ) rules = load_build_rules({"tool": {"neko": {"build": build_table}}}) - keep = module._plugin_stage_filter(tmp_path) + keep = module._plugin_stage_filter(plugin_dir.parents[2]) relatives = [ "runtime.py", "README.md", @@ -584,6 +595,8 @@ def test_plugin_stage_filter_matches_the_real_build_rules(tmp_path: Path) -> Non ".github/workflows/ci.yml", ".vscode/settings.json", ".mypy_cache/module.json", + "cache/blob.json", + "cache/nested/blob.json", "data layer/worker.py", "assets/nested/ok.png", # Not matched by any `include` pattern -> dropped by the allow-list. @@ -596,12 +609,21 @@ def test_plugin_stage_filter_matches_the_real_build_rules(tmp_path: Path) -> Non # has drifted, and the fix is the same one line. The sole documented # asymmetry is .db/.log, which staging strips afterwards rather than through # these rules, so should_skip_path has no opinion on them. + def _really_staged(relative: str) -> bool: + """Model _copy_plugin_tree: prune directories first, then judge the file.""" + parts = PurePosixPath(relative).parts + for index in range(len(parts) - 1): + ancestor = Path(*parts[: index + 1]) + if should_skip_path(ancestor, is_dir=True, rules=rules): + return False + return not should_skip_path(Path(relative), is_dir=False, rules=rules) + for relative in relatives: mirrored_keep = keep(f"plugin/plugins/demo/{relative}") if PurePosixPath(relative).suffix.lower() in {".db", ".log"}: assert not mirrored_keep, relative continue - real_keeps = not should_skip_path(Path(relative), is_dir=False, rules=rules) + real_keeps = _really_staged(relative) assert mirrored_keep == real_keeps, ( f"mirror and build rules disagree on {relative}: " f"mirror keeps={mirrored_keep}, staging keeps={real_keeps}" @@ -618,9 +640,11 @@ def test_plugin_stage_filter_matches_the_real_build_rules(tmp_path: Path) -> Non "dist/bundle.js", "__pycache__/mod.pyc", ".mypy_cache/module.json", + "cache/blob.json", "store.db", "runtime.log", - "docs/manual.md", + # NOT docs/manual.md: without an `include` allow-list it is staged, and + # the parity loop above already pins both rule shapes. ): assert not keep(f"plugin/plugins/demo/{dropped}"), dropped From 2cd4581b29c4b58f12c0b869191c94f2a6402f92 Mon Sep 17 00:00:00 2001 From: "H.W." Date: Fri, 14 Aug 2026 15:54:54 -0700 Subject: [PATCH 12/13] =?UTF-8?q?fix(ci):=20steamworg=20=E5=8E=9F=E7=94=9F?= =?UTF-8?q?=E5=BA=93=E7=9B=AE=E5=BD=95=E7=BA=B3=E5=85=A5=E6=89=AB=E6=8F=8F?= =?UTF-8?q?=E9=9D=A2?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --include-package=steamworks 会把该目录下的每个原生库当作 package data 一起 带进包（工作流注释里写着），而清理那步只删固定的错平台文件名，所以新增一个 steamworks/中文.dylib 会直接进 Contents/MacOS。该目录有 26 个 tracked 文件， 是源码树里看得见的，加进 BUNDLED_ROOTS 即可，补了测试。 Co-Authored-By: Claude Opus 5 --- scripts/check_no_nonascii_asset_names.py | 4 ++++ tests/unit/test_check_no_nonascii_asset_names.py | 14 ++++++++++++++ 2 files changed, 18 insertions(+) diff --git a/scripts/check_no_nonascii_asset_names.py b/scripts/check_no_nonascii_asset_names.py index b9f97dd15e..b8d9e59084 100644 --- a/scripts/check_no_nonascii_asset_names.py +++ b/scripts/check_no_nonascii_asset_names.py @@ -208,6 +208,10 @@ "frontend/plugin-manager/src", "frontend/react-neko-chat/src", "plugin/plugins", + # --include-package=steamworks pulls every native lib in this directory in + # as package data; the workflow's cleanup only removes the fixed + # wrong-platform filenames, so anything else here lands in the payload. + "steamworks", # Voice-turn + speaker models, both --include-data-dir'd. The .onnx weights # are downloaded at build time and gitignored, so the git listing cannot see # them; these roots cover whatever *is* tracked here, and make diff --git a/tests/unit/test_check_no_nonascii_asset_names.py b/tests/unit/test_check_no_nonascii_asset_names.py index 2992ebaed4..55a0736196 100644 --- a/tests/unit/test_check_no_nonascii_asset_names.py +++ b/tests/unit/test_check_no_nonascii_asset_names.py @@ -881,3 +881,17 @@ def test_loose_plugin_runtime_artifacts_are_not_reported(tmp_path: Path) -> None offenders, _ = module.collect_offenders(tmp_path) assert offenders == {f"plugin/plugins/{CJK_NAME}.json"} + + +@pytest.mark.unit +def test_steamworks_native_libraries_are_scanned(tmp_path: Path) -> None: + """--include-package=steamworks carries this directory in as package data.""" + module = _load_script_module() + _init_repo(tmp_path) + steamworks = tmp_path / "steamworks" + steamworks.mkdir() + (steamworks / f"{CJK_NAME}.dylib").write_bytes(b"x") + (steamworks / "SteamworksPy.dylib").write_bytes(b"x") + + offenders, _ = module.collect_offenders(tmp_path) + assert offenders == {f"steamworks/{CJK_NAME}.dylib"} From c30d60ad5b2be7f7795f28449628a936506d9ebc Mon Sep 17 00:00:00 2001 From: "H.W." Date: Fri, 14 Aug 2026 16:18:20 -0700 Subject: [PATCH 13/13] =?UTF-8?q?test:=20=E5=9B=BA=E5=AE=9A=E6=B5=8B?= =?UTF-8?q?=E8=AF=95=E4=BB=93=E5=BA=93=E7=9A=84=E9=BB=98=E8=AE=A4=E5=88=86?= =?UTF-8?q?=E6=94=AF=E5=90=8D=EF=BC=88=E4=BF=AE=20Windows=20=E5=8D=95?= =?UTF-8?q?=E6=B5=8B=E7=BA=A2=EF=BC=89?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Windows runner 上 test_baseline_growth_is_measured_at_the_merge_base 报 SystemExit: 2——不是被测逻辑的问题，是我的 fixture 假设了 `git init` 出来的 分支叫 main。init.defaultBranch 不是各处都一样，runner 上是 master，于是 _baseline_growth("main") 走到"引用不可解析"那条硬失败分支。 _init_repo 改用 git -c init.defaultBranch=main init。顺带一提， test_missing_merge_base_fails_instead_of_using_the_tip 在 runner 上其实是 "因为错误的原因通过"（它期望退 2，而 main 不存在时也退 2），这个改动把它 也拉回到验证真正的失败路径。 验证：把 HOME 指到一个 init.defaultBranch=master 的临时配置下跑整个文件， 修复前红同一条、修复后 32 条全绿。 Co-Authored-By: Claude Opus 5 --- tests/unit/test_check_no_nonascii_asset_names.py | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/tests/unit/test_check_no_nonascii_asset_names.py b/tests/unit/test_check_no_nonascii_asset_names.py index 55a0736196..d6d652f8ac 100644 --- a/tests/unit/test_check_no_nonascii_asset_names.py +++ b/tests/unit/test_check_no_nonascii_asset_names.py @@ -48,8 +48,15 @@ def _load_script_module(): def _init_repo(root: Path) -> None: - """A minimal git repo — the checker asks git which files exist.""" - subprocess.run(["git", "init", "-q"], cwd=root, check=True) + """A minimal git repo — the checker asks git which files exist. + + The branch name is pinned: `git init` honours init.defaultBranch, which is + not `main` everywhere (the Windows runner proved it), and the merge-base + tests name their base branch explicitly. + """ + subprocess.run( + ["git", "-c", "init.defaultBranch=main", "init", "-q"], cwd=root, check=True + ) # ---------------------------------------------------------------------------