Phase 23 "可信系统测试重建 (REFACTOR)" re-audit. STATUS claimed "10 E2E
files / ~80+ cases" and "INV-011~020 ✅"; measurement showed the opposite.
D-23-A (Critical): V11InvariantsTest's 14 cases asserted only Constants
strings. INV-016 "Banner target types" actually asserted ARTICLE_STATUS_*;
INV-017 "home 8 entries" declared local arrays then asserted
firstRow.length == 4 (tautology, never called the home service);
INV-018/019/020 used appointment/article/account status constants as
stand-ins for permission and data-scope checks. Rewritten to 12 cases:
4 real service-behavior guards (INV-013 via real CounselingService.getDetail
— Matchmaker sees null userRequestSummary, with an assigned-Counselor
counter-proof so global sanitization cannot fake a pass; INV-016 reads the
production ALLOWED_TARGET_TYPES whitelist and asserts no external/http/url/
web entry; INV-017 calls real getHome and pins all 8 codes + order;
INV-019 asserts content_editor reads null), plus assertBehaviorTestExists
turning "行为验证见 XxxTest" pointers into executable Class.forName contracts.
D-23-B (High): 44 `pass` shells across 6 files in tests/e2e (4 files were
100% shells). Gate forbids skip/pass shells. Rewritten in place as honest
placeholders with the full scenario lists preserved.
D-23-C (High, residual): the pytest E2E suite is not executable at all —
no conftest.py anywhere, ~15 fixtures (api_client/verified_user/
banner_factory/...) do not exist. Only run_v11_tests.py really runs
(16 cases, needs a live stack + E2E_PASSWORD).
D-23-D (High, residual, NOT fixed): CI does not chain E2E as the plan
requires. Needs an infrastructure decision on how CI provisions a live
stack; not implemented rather than written unverified.
D-23-E (Medium): the CI migration job claimed to "验证 migration SQL 语法"
but only checked file non-emptiness, while the real syntax/baseline checks
live in MigrationValidationTest (already run by the backend job). Replaced
with a real sequence check (naming / non-empty / unique versions / no
gaps); verified by extracting the exact inline script and running it.
New gate: scripts/check-no-empty-tests.py (AST scan, blocks pass-only and
unconditionally skipped tests) + a CI test-shell-guard job. Verified both
ways: 0 findings on the real tree, 2 precise findings on injected shells.
Negative validation: 3 injected production defects (external_url in the
banner whitelist, swapped home entry order, isMatchmaker returning the full
record) turned exactly inv016/inv017/inv013 red, 9 others green, zero
unrelated failures; md5sum -c confirmed byte-identical restore.
Full regression: 655 tests / 652 passed / 0 failures / 0 errors / 3 skipped
(baseline 657; -2 because 14 vacuous cases were merged into 12). The 3
skips carry explicit reasons (2x @EnabledIfEnvironmentVariable(CI=true),
1x @Disabled "需要完整数据库连接") — none is a shell.
Phase 23 is PARTIALLY closed: fake coverage is gone, but the UAT gate
requires the end-to-end chain to actually execute, so "失败为零可进入 UAT"
is NOT claimed.
138 lines
4.8 KiB
Python
138 lines
4.8 KiB
Python
#!/usr/bin/env python3
|
||
"""
|
||
Phase 23 Gate 守卫:禁止测试空壳(skip / pass 空壳)。
|
||
|
||
计划 Phase 23 Gate 原文:
|
||
「不允许 skip/pass 空壳;报告列出总数、通过、失败、跳过及原因,失败为零才进入 UAT。」
|
||
|
||
背景(Phase 23 KEEP-VERIFY 实测)
|
||
--------------------------------
|
||
`tests/e2e/` 下曾有 **44 个**函数体只有 `pass` 的测试函数,分属 6 个文件,
|
||
其中 4 个文件 100% 由空壳构成;这些用例从未执行过(且依赖根本不存在的 pytest fixture),
|
||
却在 `docs/STATUS.md` 中被计为 P23-Txx ✅。
|
||
本守卫用 AST(而非 grep)在 CI 中机械阻断空壳回归。
|
||
|
||
检测项
|
||
------
|
||
1. 函数体为空、或仅为「docstring + pass」的函数(测试函数与非测试的 helper 均计入,
|
||
因为空壳 helper 同样是未实现逻辑);
|
||
2. 被 `@pytest.mark.skip` / `skipif` 无理由装饰、或函数体内直接
|
||
`pytest.skip(...)` 开头跳过的「测试函数」(`test_*`)。
|
||
|
||
用法
|
||
----
|
||
python3 scripts/check-no-empty-tests.py [目录 ...] # 默认扫描 tests/
|
||
|
||
退出码:0 = 无空壳;1 = 发现空壳(并打印 文件:行 清单)。
|
||
"""
|
||
|
||
from __future__ import annotations
|
||
|
||
import ast
|
||
import sys
|
||
from pathlib import Path
|
||
|
||
SKIP_DECORATORS = {"skip", "skipif"}
|
||
|
||
|
||
def is_docstring_only_then_pass(node: ast.FunctionDef | ast.AsyncFunctionDef) -> bool:
|
||
"""函数体是否为空,或仅为「docstring + pass」。"""
|
||
body = [stmt for stmt in node.body]
|
||
if not body:
|
||
return True
|
||
# 去掉开头的 docstring
|
||
if (
|
||
isinstance(body[0], ast.Expr)
|
||
and isinstance(body[0].value, ast.Constant)
|
||
and isinstance(body[0].value.value, str)
|
||
):
|
||
body = body[1:]
|
||
if not body:
|
||
return True
|
||
return all(isinstance(stmt, ast.Pass) for stmt in body)
|
||
|
||
|
||
def has_skip_marker(node: ast.FunctionDef | ast.AsyncFunctionDef) -> bool:
|
||
"""是否被无条件 skip 的装饰器标记。"""
|
||
for dec in node.decorator_list:
|
||
target = None
|
||
if isinstance(dec, ast.Attribute):
|
||
target = dec.attr
|
||
elif isinstance(dec, ast.Call) and isinstance(dec.func, ast.Attribute):
|
||
target = dec.func.attr
|
||
if target in SKIP_DECORATORS and not node.name.startswith("test_"):
|
||
continue
|
||
if target in SKIP_DECORATORS:
|
||
return True
|
||
return False
|
||
|
||
|
||
def starts_with_pytest_skip(node: ast.FunctionDef | ast.AsyncFunctionDef) -> bool:
|
||
"""函数体是否以 pytest.skip(...) 打头(除 docstring 外)。"""
|
||
body = list(node.body)
|
||
if (
|
||
body
|
||
and isinstance(body[0], ast.Expr)
|
||
and isinstance(body[0].value, ast.Constant)
|
||
and isinstance(body[0].value.value, str)
|
||
):
|
||
body = body[1:]
|
||
if not body:
|
||
return False
|
||
stmt = body[0]
|
||
if not isinstance(stmt, ast.Expr) or not isinstance(stmt.value, ast.Call):
|
||
return False
|
||
func = stmt.value.func
|
||
return isinstance(func, ast.Attribute) and func.attr == "skip"
|
||
|
||
|
||
def scan_file(path: Path) -> list[str]:
|
||
try:
|
||
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
|
||
except SyntaxError as exc: # 语法错误交给 pytest 自己去报,这里不重复处理
|
||
return [f"{path}:{exc.lineno or 0}: 无法解析(SyntaxError: {exc.msg})"]
|
||
|
||
findings: list[str] = []
|
||
for node in ast.walk(tree):
|
||
if not isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
|
||
continue
|
||
if is_docstring_only_then_pass(node):
|
||
findings.append(
|
||
f"{path}:{node.lineno}: 空壳函数 `{node.name}`(函数体只有 pass / 为空)"
|
||
)
|
||
elif node.name.startswith("test_") and (
|
||
has_skip_marker(node) or starts_with_pytest_skip(node)
|
||
):
|
||
findings.append(
|
||
f"{path}:{node.lineno}: 测试 `{node.name}` 被无条件 skip(Gate 禁止 skip 空壳)"
|
||
)
|
||
return findings
|
||
|
||
|
||
def main(argv: list[str]) -> int:
|
||
roots = [Path(a) for a in argv[1:]] or [Path("tests")]
|
||
files: list[Path] = []
|
||
for root in roots:
|
||
if root.is_file():
|
||
files.append(root)
|
||
else:
|
||
files.extend(sorted(root.rglob("*.py")))
|
||
|
||
all_findings: list[str] = []
|
||
for f in files:
|
||
all_findings.extend(scan_file(f))
|
||
|
||
if all_findings:
|
||
print("FAIL: 发现测试空壳(Phase 23 Gate 禁止 skip/pass 空壳):", file=sys.stderr)
|
||
for line in all_findings:
|
||
print(" " + line, file=sys.stderr)
|
||
print(f"\n合计 {len(all_findings)} 处。请实现真实断言,或删除该空壳。", file=sys.stderr)
|
||
return 1
|
||
|
||
print(f"OK: 未发现 skip/pass 空壳(已扫描 {len(files)} 个 Python 文件)")
|
||
return 0
|
||
|
||
|
||
if __name__ == "__main__":
|
||
raise SystemExit(main(sys.argv))
|