Files
danyang-match/scripts/check-no-empty-tests.py
T
WorkBuddy ae1bd2b734 test(phase23): KEEP-VERIFY — remove 44 pass shells + fake INV tests, add shell guard
Phase 23 "可信系统测试重建 (REFACTOR)" re-audit. STATUS claimed "10 E2E
files / ~80+ cases" and "INV-011~020 ✅"; measurement showed the opposite.

D-23-A (Critical): V11InvariantsTest's 14 cases asserted only Constants
strings. INV-016 "Banner target types" actually asserted ARTICLE_STATUS_*;
INV-017 "home 8 entries" declared local arrays then asserted
firstRow.length == 4 (tautology, never called the home service);
INV-018/019/020 used appointment/article/account status constants as
stand-ins for permission and data-scope checks. Rewritten to 12 cases:
4 real service-behavior guards (INV-013 via real CounselingService.getDetail
— Matchmaker sees null userRequestSummary, with an assigned-Counselor
counter-proof so global sanitization cannot fake a pass; INV-016 reads the
production ALLOWED_TARGET_TYPES whitelist and asserts no external/http/url/
web entry; INV-017 calls real getHome and pins all 8 codes + order;
INV-019 asserts content_editor reads null), plus assertBehaviorTestExists
turning "行为验证见 XxxTest" pointers into executable Class.forName contracts.

D-23-B (High): 44 `pass` shells across 6 files in tests/e2e (4 files were
100% shells). Gate forbids skip/pass shells. Rewritten in place as honest
placeholders with the full scenario lists preserved.

D-23-C (High, residual): the pytest E2E suite is not executable at all —
no conftest.py anywhere, ~15 fixtures (api_client/verified_user/
banner_factory/...) do not exist. Only run_v11_tests.py really runs
(16 cases, needs a live stack + E2E_PASSWORD).

D-23-D (High, residual, NOT fixed): CI does not chain E2E as the plan
requires. Needs an infrastructure decision on how CI provisions a live
stack; not implemented rather than written unverified.

D-23-E (Medium): the CI migration job claimed to "验证 migration SQL 语法"
but only checked file non-emptiness, while the real syntax/baseline checks
live in MigrationValidationTest (already run by the backend job). Replaced
with a real sequence check (naming / non-empty / unique versions / no
gaps); verified by extracting the exact inline script and running it.

New gate: scripts/check-no-empty-tests.py (AST scan, blocks pass-only and
unconditionally skipped tests) + a CI test-shell-guard job. Verified both
ways: 0 findings on the real tree, 2 precise findings on injected shells.

Negative validation: 3 injected production defects (external_url in the
banner whitelist, swapped home entry order, isMatchmaker returning the full
record) turned exactly inv016/inv017/inv013 red, 9 others green, zero
unrelated failures; md5sum -c confirmed byte-identical restore.

Full regression: 655 tests / 652 passed / 0 failures / 0 errors / 3 skipped
(baseline 657; -2 because 14 vacuous cases were merged into 12). The 3
skips carry explicit reasons (2x @EnabledIfEnvironmentVariable(CI=true),
1x @Disabled "需要完整数据库连接") — none is a shell.

Phase 23 is PARTIALLY closed: fake coverage is gone, but the UAT gate
requires the end-to-end chain to actually execute, so "失败为零可进入 UAT"
is NOT claimed.
2026-09-18 11:49:18 +08:00

138 lines
4.8 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
#!/usr/bin/env python3
"""
Phase 23 Gate 守卫:禁止测试空壳(skip / pass 空壳)。
计划 Phase 23 Gate 原文:
「不允许 skip/pass 空壳;报告列出总数、通过、失败、跳过及原因,失败为零才进入 UAT。」
背景(Phase 23 KEEP-VERIFY 实测)
--------------------------------
`tests/e2e/` 下曾有 **44 个**函数体只有 `pass` 的测试函数,分属 6 个文件,
其中 4 个文件 100% 由空壳构成;这些用例从未执行过(且依赖根本不存在的 pytest fixture),
却在 `docs/STATUS.md` 中被计为 P23-Txx ✅。
本守卫用 AST(而非 grep)在 CI 中机械阻断空壳回归。
检测项
------
1. 函数体为空、或仅为「docstring + pass」的函数(测试函数与非测试的 helper 均计入,
因为空壳 helper 同样是未实现逻辑);
2. 被 `@pytest.mark.skip` / `skipif` 无理由装饰、或函数体内直接
`pytest.skip(...)` 开头跳过的「测试函数」(`test_*`)。
用法
----
python3 scripts/check-no-empty-tests.py [目录 ...] # 默认扫描 tests/
退出码:0 = 无空壳;1 = 发现空壳(并打印 文件:行 清单)。
"""
from __future__ import annotations
import ast
import sys
from pathlib import Path
SKIP_DECORATORS = {"skip", "skipif"}
def is_docstring_only_then_pass(node: ast.FunctionDef | ast.AsyncFunctionDef) -> bool:
"""函数体是否为空,或仅为「docstring + pass」。"""
body = [stmt for stmt in node.body]
if not body:
return True
# 去掉开头的 docstring
if (
isinstance(body[0], ast.Expr)
and isinstance(body[0].value, ast.Constant)
and isinstance(body[0].value.value, str)
):
body = body[1:]
if not body:
return True
return all(isinstance(stmt, ast.Pass) for stmt in body)
def has_skip_marker(node: ast.FunctionDef | ast.AsyncFunctionDef) -> bool:
"""是否被无条件 skip 的装饰器标记。"""
for dec in node.decorator_list:
target = None
if isinstance(dec, ast.Attribute):
target = dec.attr
elif isinstance(dec, ast.Call) and isinstance(dec.func, ast.Attribute):
target = dec.func.attr
if target in SKIP_DECORATORS and not node.name.startswith("test_"):
continue
if target in SKIP_DECORATORS:
return True
return False
def starts_with_pytest_skip(node: ast.FunctionDef | ast.AsyncFunctionDef) -> bool:
"""函数体是否以 pytest.skip(...) 打头(除 docstring 外)。"""
body = list(node.body)
if (
body
and isinstance(body[0], ast.Expr)
and isinstance(body[0].value, ast.Constant)
and isinstance(body[0].value.value, str)
):
body = body[1:]
if not body:
return False
stmt = body[0]
if not isinstance(stmt, ast.Expr) or not isinstance(stmt.value, ast.Call):
return False
func = stmt.value.func
return isinstance(func, ast.Attribute) and func.attr == "skip"
def scan_file(path: Path) -> list[str]:
try:
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
except SyntaxError as exc: # 语法错误交给 pytest 自己去报,这里不重复处理
return [f"{path}:{exc.lineno or 0}: 无法解析(SyntaxError: {exc.msg})"]
findings: list[str] = []
for node in ast.walk(tree):
if not isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
continue
if is_docstring_only_then_pass(node):
findings.append(
f"{path}:{node.lineno}: 空壳函数 `{node.name}`(函数体只有 pass / 为空)"
)
elif node.name.startswith("test_") and (
has_skip_marker(node) or starts_with_pytest_skip(node)
):
findings.append(
f"{path}:{node.lineno}: 测试 `{node.name}` 被无条件 skip(Gate 禁止 skip 空壳)"
)
return findings
def main(argv: list[str]) -> int:
roots = [Path(a) for a in argv[1:]] or [Path("tests")]
files: list[Path] = []
for root in roots:
if root.is_file():
files.append(root)
else:
files.extend(sorted(root.rglob("*.py")))
all_findings: list[str] = []
for f in files:
all_findings.extend(scan_file(f))
if all_findings:
print("FAIL: 发现测试空壳(Phase 23 Gate 禁止 skip/pass 空壳):", file=sys.stderr)
for line in all_findings:
print(" " + line, file=sys.stderr)
print(f"\n合计 {len(all_findings)} 处。请实现真实断言,或删除该空壳。", file=sys.stderr)
return 1
print(f"OK: 未发现 skip/pass 空壳(已扫描 {len(files)} 个 Python 文件)")
return 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv))