Hongming Wang 26fa220bef ci(coverage): per-file 75% floor for MCP/inbox/auth Python critical paths

Closes part of #2790 (Phase A). The Python total floor at 86% (set in
workspace/pytest.ini, issue #1817) averages over ~6000 lines, so a
single MCP-critical file could regress to ~50% with no CI complaint as
long as other modules compensate. This is the same distribution gap
that #1823 closed Go-side: total floor passes while a critical handler
sits at 0%.

Added gates for these five files (per-file floor 75%):
- workspace/a2a_mcp_server.py — MCP dispatcher (PR #2766 / #2771)
- workspace/mcp_cli.py — molecule-mcp standalone CLI entry
- workspace/a2a_tools.py — workspace-scoped tool implementations
- workspace/inbox.py — multi-workspace inbox + per-workspace cursors
- workspace/platform_auth.py — per-workspace token resolver

These handle multi-tenant routing, auth tokens, and inbox dispatch.
Risk shape mirrors Go-side tokens*/secrets* — a 0%/50% file here is
exactly where the PR #2766 dispatcher bug class slips through without
a structural test.

Floor 75% is strictly additive — current actuals 80-96% (measured
2026-05-04). No existing PR fails. Ratchet plan in COVERAGE_FLOOR.md
target 90% by 2026-08-04.

Implementation: pytest already writes .coverage; new step emits a JSON
view scoped to the critical files via `coverage json --include="*name"`,
then jq extracts each file's percent_covered. Exact key match by
basename so workspace/builtin_tools/a2a_tools.py (a different 100%
file) doesn't shadow workspace/a2a_tools.py.

Verified locally with the actual coverage data:
- floor=75 → 0 failures (matches current state)
- floor=81 → 1 failure (a2a_tools.py at 80%) — proves the gate trips

Pairs with PR #2791 (Phase B — schema↔dispatcher AST drift gate). Phase
C (molecule-mcp e2e harness) remains the largest piece in #2790.

YAML validated locally before commit per
feedback_validate_yaml_before_commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-04 16:35:21 -07:00

5.0 KiB

Raw Permalink Blame History

Coverage Floor

CI enforces coverage gates on two surfaces — workspace-server (Go) and workspace/ (Python). All defined in .github/workflows/ci.yml.

Current floors (2026-04-23)

Gate	Threshold	What fails
Total floor	`25%`	`go tool cover -func` reports total below floor
Critical-path per-file floor	`10%`	Any non-test source file in a security-critical path with coverage ≤10%
Per-file report	advisory	Printed in CI log, sorted worst-first, does not fail

Total floor starts at 25% (unchanged from pre-#1823 to keep this PR strictly additive). The new protection is the critical-path per-file floor, which directly closes the gap that prompted the issue. Ratchet plan below begins the month after to let the team first observe the gate in action.

Security-critical paths (Gate 2)

Changes to these paths have historically introduced security issues (CWE-22, CWE-78, KI-005, SSRF) or billing/auth risk. Coverage must not drop to zero.

internal/handlers/tokens*
internal/handlers/workspace_provision*
internal/handlers/a2a_proxy*
internal/handlers/registry*
internal/handlers/secrets*
internal/middleware/wsauth*
internal/crypto*

Ratchet plan

Floor ratchets upward on a fixed cadence. Any ratchet is a PR — reviewable, reversible, and creates history. The table below is the intended schedule.

Date	Total floor	Critical-path floor	Notes
2026-04-23	25%	10%	Initial gate (this file).
2026-05-23	30%	20%	First ratchet
2026-06-23	40%	30%
2026-07-23	50%	40%
2026-08-23	55%	50%
2026-09-23	60%	60%
2026-10-23	70%	70%	Target steady-state

The target end-state matches the per-role QA prompts which specify "coverage >80% on changed files". CI enforces the floor; reviewers still enforce the per-PR bar.

Exceptions

If a critical-path file genuinely cannot have coverage above the floor (e.g. thin wrapper around a third-party SDK with no branches to test), add an entry here with:

File: internal/handlers/example.go
Reason: Why coverage can't hit the floor
Tracking issue: GitHub issue for the real fix
Expiry: 14 days from entry date; after expiry either coverage is fixed or the issue is closed as "accepted technical debt"

Active exceptions

(none — add here if you need to land code that legitimately can't clear the floor)

Why this gate exists

Issue #1823: an external audit found critical files at 0% coverage despite test files existing with hundreds of lines. The existing CI step measured coverage but didn't enforce a meaningful threshold. Any file could go from 80% → 0% and CI stayed green, because the single gate (total ≥25%) ignored per-file distribution.

This gate makes "no untested critical paths merged" a mechanical property of the CI, not a behavioural property of QA agents or individual reviewers — which is the only way to make it survive fleet outages, agent rotations, or QA process changes.

Python (workspace/) — added 2026-05-04 from #2790

The Python side has its own gates in the python-lint job:

Gate	Threshold	Where
Total floor	`86%`	`workspace/pytest.ini` `--cov-fail-under=86` (issue #1817)
Critical-path per-file floor	`75%`	Inline shell step after the pytest run

Critical-path Python files

These handle multi-tenant routing, auth tokens, and inbox dispatch. A coverage drop here is the same risk shape as a Go-side tokens* / secrets* file regressing below 10%.

workspace/a2a_mcp_server.py — MCP dispatcher (PR #2766 / #2771)
workspace/mcp_cli.py — molecule-mcp standalone CLI entry
workspace/a2a_tools.py — workspace-scoped tool implementations
workspace/inbox.py — multi-workspace inbox + per-workspace cursors
workspace/platform_auth.py — per-workspace token resolver

Why 75% (vs 86% total)

The total floor averages ~6000 lines across workspace/. A single MCP file could drop to ~50% with no CI complaint as long as other modules compensate. The per-file floor closes that distribution gap. 75% sits below current actuals (80–96% as of 2026-05-04) — strictly additive, no existing PR fails.

Python ratchet plan

Date	Total	Per-file critical	Notes
2026-05-04	86%	75%	Initial gate (this file).
2026-06-04	86%	80%	First ratchet — at-floor files must catch up.
2026-07-04	88%	85%
2026-08-04	90%	90%	Target steady-state.

Why this Python gate exists

Issue #2790, after the PR #2766 → PR #2771 cycle. PR #2766 added multi-workspace routing through a2a_tools.py + a2a_mcp_server.py, shipped to main with green CI, but the dispatcher silently dropped a load-bearing kwarg for 4 of 9 tools — caught only by post-merge code review. The structural drift gate (test_dispatcher_schema_drift.py, PR #2791) catches the schema↔dispatcher mismatch class; this floor catches the broader "MCP-critical file regressed" class.

5.0 KiB Raw Permalink Blame History Unescape Escape