From 26fa220befe5be0b2a3e506f4ab82f41e6c97cdd Mon Sep 17 00:00:00 2001 From: Hongming Wang Date: Mon, 4 May 2026 16:35:21 -0700 Subject: [PATCH] ci(coverage): per-file 75% floor for MCP/inbox/auth Python critical paths MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes part of #2790 (Phase A). The Python total floor at 86% (set in workspace/pytest.ini, issue #1817) averages over ~6000 lines, so a single MCP-critical file could regress to ~50% with no CI complaint as long as other modules compensate. This is the same distribution gap that #1823 closed Go-side: total floor passes while a critical handler sits at 0%. Added gates for these five files (per-file floor 75%): - workspace/a2a_mcp_server.py — MCP dispatcher (PR #2766 / #2771) - workspace/mcp_cli.py — molecule-mcp standalone CLI entry - workspace/a2a_tools.py — workspace-scoped tool implementations - workspace/inbox.py — multi-workspace inbox + per-workspace cursors - workspace/platform_auth.py — per-workspace token resolver These handle multi-tenant routing, auth tokens, and inbox dispatch. Risk shape mirrors Go-side tokens*/secrets* — a 0%/50% file here is exactly where the PR #2766 dispatcher bug class slips through without a structural test. Floor 75% is strictly additive — current actuals 80-96% (measured 2026-05-04). No existing PR fails. Ratchet plan in COVERAGE_FLOOR.md target 90% by 2026-08-04. Implementation: pytest already writes .coverage; new step emits a JSON view scoped to the critical files via `coverage json --include="*name"`, then jq extracts each file's percent_covered. Exact key match by basename so workspace/builtin_tools/a2a_tools.py (a different 100% file) doesn't shadow workspace/a2a_tools.py. Verified locally with the actual coverage data: - floor=75 → 0 failures (matches current state) - floor=81 → 1 failure (a2a_tools.py at 80%) — proves the gate trips Pairs with PR #2791 (Phase B — schema↔dispatcher AST drift gate). Phase C (molecule-mcp e2e harness) remains the largest piece in #2790. YAML validated locally before commit per feedback_validate_yaml_before_commit. Co-Authored-By: Claude Opus 4.7 (1M context) --- .github/workflows/ci.yml | 66 ++++++++++++++++++++++++++++++++++++++++ COVERAGE_FLOOR.md | 52 +++++++++++++++++++++++++++++-- 2 files changed, 116 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7f0c72bb..a642e1a3 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -358,6 +358,72 @@ jobs: - if: needs.changes.outputs.python == 'true' run: python -m pytest --tb=short + - if: needs.changes.outputs.python == 'true' + name: Per-file critical-path coverage (MCP / inbox / auth) + # MCP-critical Python files have a per-file floor on top of the + # 86% total floor in pytest.ini. Rationale (issue #2790, after + # the PR #2766 → PR #2771 cycle): the total floor averages ~6000 + # lines, so a single MCP file could regress to ~50% with no + # complaint as long as other modules compensate. These five + # files handle multi-tenant routing + auth + inbox dispatch — + # a coverage drop here is the same risk shape as a Go-side + # workspace-server token/secrets file dropping below 10%. + # + # Floor 75% sits below current actuals (80-96%) so this gate is + # strictly additive — no existing PR fails. Ratchet plan in + # COVERAGE_FLOOR.md. + run: | + set -e + PER_FILE_FLOOR=75 + CRITICAL_FILES=( + "a2a_mcp_server.py" + "mcp_cli.py" + "a2a_tools.py" + "inbox.py" + "platform_auth.py" + ) + + # pytest already wrote .coverage; emit a JSON view scoped to + # the critical files so jq/python can read the per-file pct + # without parsing tabular text. --include uses fnmatch, and + # the leading "*" allows the file to live anywhere under the + # workspace root (today they sit at workspace/.py). + INCLUDES=$(printf '*%s,' "${CRITICAL_FILES[@]}") + INCLUDES="${INCLUDES%,}" + python -m coverage json -o /tmp/critical-cov.json --include="$INCLUDES" + + FAILED=0 + for f in "${CRITICAL_FILES[@]}"; do + # Match by top-level path key (e.g. "a2a_tools.py", not + # "builtin_tools/a2a_tools.py" — different file at 100%). + # The keys in coverage.json are paths relative to the run + # cwd (workspace/), so the critical-path entry sits at the + # bare basename. + pct=$(jq -r --arg f "$f" '.files | to_entries | map(select(.key == $f)) | .[0].value.summary.percent_covered // "MISSING"' /tmp/critical-cov.json) + if [ "$pct" = "MISSING" ]; then + echo "::error file=workspace/$f::No coverage data — file may have moved or test exclusion mis-set." + FAILED=$((FAILED+1)) + continue + fi + echo "$f: ${pct}%" + if awk "BEGIN{exit !($pct < $PER_FILE_FLOOR)}"; then + echo "::error file=workspace/$f::${pct}% < ${PER_FILE_FLOOR}% per-file floor (MCP critical path). See COVERAGE_FLOOR.md." + FAILED=$((FAILED+1)) + fi + done + + if [ "$FAILED" -gt 0 ]; then + echo "" + echo "$FAILED MCP critical-path file(s) below the ${PER_FILE_FLOOR}% per-file floor." + echo "These paths handle multi-tenant routing, auth tokens, and inbox dispatch." + echo "A coverage drop here is the same risk shape as Go-side tokens/secrets files" + echo "dropping below 10% (see COVERAGE_FLOOR.md). Either:" + echo " (a) add tests to raise coverage back above ${PER_FILE_FLOOR}%, or" + echo " (b) if this is unavoidable historical debt, file an issue and propose" + echo " adjusting the floor with rationale in COVERAGE_FLOOR.md." + exit 1 + fi + # SDK + plugin validation moved to standalone repo: # github.com/Molecule-AI/molecule-sdk-python diff --git a/COVERAGE_FLOOR.md b/COVERAGE_FLOOR.md index 2870a649..c768b318 100644 --- a/COVERAGE_FLOOR.md +++ b/COVERAGE_FLOOR.md @@ -1,7 +1,7 @@ # Coverage Floor -CI enforces three coverage gates on `workspace-server` (Go). All defined in -`.github/workflows/ci.yml` → `platform-build` job. +CI enforces coverage gates on two surfaces — `workspace-server` (Go) and +`workspace/` (Python). All defined in `.github/workflows/ci.yml`. ## Current floors (2026-04-23) @@ -76,3 +76,51 @@ This gate makes "no untested critical paths merged" a mechanical property of the CI, not a behavioural property of QA agents or individual reviewers — which is the only way to make it survive fleet outages, agent rotations, or QA process changes. + +## Python (workspace/) — added 2026-05-04 from #2790 + +The Python side has its own gates in the `python-lint` job: + +| Gate | Threshold | Where | +|---|---|---| +| **Total floor** | `86%` | `workspace/pytest.ini` `--cov-fail-under=86` (issue #1817) | +| **Critical-path per-file floor** | `75%` | Inline shell step after the pytest run | + +### Critical-path Python files + +These handle multi-tenant routing, auth tokens, and inbox dispatch. A +coverage drop here is the same risk shape as a Go-side `tokens*` / +`secrets*` file regressing below 10%. + +- `workspace/a2a_mcp_server.py` — MCP dispatcher (PR #2766 / #2771) +- `workspace/mcp_cli.py` — molecule-mcp standalone CLI entry +- `workspace/a2a_tools.py` — workspace-scoped tool implementations +- `workspace/inbox.py` — multi-workspace inbox + per-workspace cursors +- `workspace/platform_auth.py` — per-workspace token resolver + +### Why 75% (vs 86% total) + +The total floor averages ~6000 lines across `workspace/`. A single MCP +file could drop to ~50% with no CI complaint as long as other modules +compensate. The per-file floor closes that distribution gap. 75% sits +below current actuals (80–96% as of 2026-05-04) — strictly additive, +no existing PR fails. + +### Python ratchet plan + +| Date | Total | Per-file critical | Notes | +|---|---|---|---| +| 2026-05-04 | 86% | 75% | Initial gate (this file). | +| 2026-06-04 | 86% | 80% | First ratchet — at-floor files must catch up. | +| 2026-07-04 | 88% | 85% | | +| 2026-08-04 | 90% | 90% | Target steady-state. | + +### Why this Python gate exists + +Issue #2790, after the PR #2766 → PR #2771 cycle. PR #2766 added +multi-workspace routing through `a2a_tools.py` + `a2a_mcp_server.py`, +shipped to main with green CI, but the dispatcher silently dropped a +load-bearing kwarg for 4 of 9 tools — caught only by post-merge code +review. The structural drift gate (`test_dispatcher_schema_drift.py`, +PR #2791) catches the schema↔dispatcher mismatch class; this floor +catches the broader "MCP-critical file regressed" class.