ci(coverage): per-file 75% floor for MCP/inbox/auth Python critical paths
Closes part of #2790 (Phase A). The Python total floor at 86% (set in workspace/pytest.ini, issue #1817) averages over ~6000 lines, so a single MCP-critical file could regress to ~50% with no CI complaint as long as other modules compensate. This is the same distribution gap that #1823 closed Go-side: total floor passes while a critical handler sits at 0%. Added gates for these five files (per-file floor 75%): - workspace/a2a_mcp_server.py — MCP dispatcher (PR #2766 / #2771) - workspace/mcp_cli.py — molecule-mcp standalone CLI entry - workspace/a2a_tools.py — workspace-scoped tool implementations - workspace/inbox.py — multi-workspace inbox + per-workspace cursors - workspace/platform_auth.py — per-workspace token resolver These handle multi-tenant routing, auth tokens, and inbox dispatch. Risk shape mirrors Go-side tokens*/secrets* — a 0%/50% file here is exactly where the PR #2766 dispatcher bug class slips through without a structural test. Floor 75% is strictly additive — current actuals 80-96% (measured 2026-05-04). No existing PR fails. Ratchet plan in COVERAGE_FLOOR.md target 90% by 2026-08-04. Implementation: pytest already writes .coverage; new step emits a JSON view scoped to the critical files via `coverage json --include="*name"`, then jq extracts each file's percent_covered. Exact key match by basename so workspace/builtin_tools/a2a_tools.py (a different 100% file) doesn't shadow workspace/a2a_tools.py. Verified locally with the actual coverage data: - floor=75 → 0 failures (matches current state) - floor=81 → 1 failure (a2a_tools.py at 80%) — proves the gate trips Pairs with PR #2791 (Phase B — schema↔dispatcher AST drift gate). Phase C (molecule-mcp e2e harness) remains the largest piece in #2790. YAML validated locally before commit per feedback_validate_yaml_before_commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
6d7a7fc86f
commit
26fa220bef
66
.github/workflows/ci.yml
vendored
66
.github/workflows/ci.yml
vendored
@ -358,6 +358,72 @@ jobs:
|
|||||||
- if: needs.changes.outputs.python == 'true'
|
- if: needs.changes.outputs.python == 'true'
|
||||||
run: python -m pytest --tb=short
|
run: python -m pytest --tb=short
|
||||||
|
|
||||||
|
- if: needs.changes.outputs.python == 'true'
|
||||||
|
name: Per-file critical-path coverage (MCP / inbox / auth)
|
||||||
|
# MCP-critical Python files have a per-file floor on top of the
|
||||||
|
# 86% total floor in pytest.ini. Rationale (issue #2790, after
|
||||||
|
# the PR #2766 → PR #2771 cycle): the total floor averages ~6000
|
||||||
|
# lines, so a single MCP file could regress to ~50% with no
|
||||||
|
# complaint as long as other modules compensate. These five
|
||||||
|
# files handle multi-tenant routing + auth + inbox dispatch —
|
||||||
|
# a coverage drop here is the same risk shape as a Go-side
|
||||||
|
# workspace-server token/secrets file dropping below 10%.
|
||||||
|
#
|
||||||
|
# Floor 75% sits below current actuals (80-96%) so this gate is
|
||||||
|
# strictly additive — no existing PR fails. Ratchet plan in
|
||||||
|
# COVERAGE_FLOOR.md.
|
||||||
|
run: |
|
||||||
|
set -e
|
||||||
|
PER_FILE_FLOOR=75
|
||||||
|
CRITICAL_FILES=(
|
||||||
|
"a2a_mcp_server.py"
|
||||||
|
"mcp_cli.py"
|
||||||
|
"a2a_tools.py"
|
||||||
|
"inbox.py"
|
||||||
|
"platform_auth.py"
|
||||||
|
)
|
||||||
|
|
||||||
|
# pytest already wrote .coverage; emit a JSON view scoped to
|
||||||
|
# the critical files so jq/python can read the per-file pct
|
||||||
|
# without parsing tabular text. --include uses fnmatch, and
|
||||||
|
# the leading "*" allows the file to live anywhere under the
|
||||||
|
# workspace root (today they sit at workspace/<name>.py).
|
||||||
|
INCLUDES=$(printf '*%s,' "${CRITICAL_FILES[@]}")
|
||||||
|
INCLUDES="${INCLUDES%,}"
|
||||||
|
python -m coverage json -o /tmp/critical-cov.json --include="$INCLUDES"
|
||||||
|
|
||||||
|
FAILED=0
|
||||||
|
for f in "${CRITICAL_FILES[@]}"; do
|
||||||
|
# Match by top-level path key (e.g. "a2a_tools.py", not
|
||||||
|
# "builtin_tools/a2a_tools.py" — different file at 100%).
|
||||||
|
# The keys in coverage.json are paths relative to the run
|
||||||
|
# cwd (workspace/), so the critical-path entry sits at the
|
||||||
|
# bare basename.
|
||||||
|
pct=$(jq -r --arg f "$f" '.files | to_entries | map(select(.key == $f)) | .[0].value.summary.percent_covered // "MISSING"' /tmp/critical-cov.json)
|
||||||
|
if [ "$pct" = "MISSING" ]; then
|
||||||
|
echo "::error file=workspace/$f::No coverage data — file may have moved or test exclusion mis-set."
|
||||||
|
FAILED=$((FAILED+1))
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
echo "$f: ${pct}%"
|
||||||
|
if awk "BEGIN{exit !($pct < $PER_FILE_FLOOR)}"; then
|
||||||
|
echo "::error file=workspace/$f::${pct}% < ${PER_FILE_FLOOR}% per-file floor (MCP critical path). See COVERAGE_FLOOR.md."
|
||||||
|
FAILED=$((FAILED+1))
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
if [ "$FAILED" -gt 0 ]; then
|
||||||
|
echo ""
|
||||||
|
echo "$FAILED MCP critical-path file(s) below the ${PER_FILE_FLOOR}% per-file floor."
|
||||||
|
echo "These paths handle multi-tenant routing, auth tokens, and inbox dispatch."
|
||||||
|
echo "A coverage drop here is the same risk shape as Go-side tokens/secrets files"
|
||||||
|
echo "dropping below 10% (see COVERAGE_FLOOR.md). Either:"
|
||||||
|
echo " (a) add tests to raise coverage back above ${PER_FILE_FLOOR}%, or"
|
||||||
|
echo " (b) if this is unavoidable historical debt, file an issue and propose"
|
||||||
|
echo " adjusting the floor with rationale in COVERAGE_FLOOR.md."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
# SDK + plugin validation moved to standalone repo:
|
# SDK + plugin validation moved to standalone repo:
|
||||||
# github.com/Molecule-AI/molecule-sdk-python
|
# github.com/Molecule-AI/molecule-sdk-python
|
||||||
|
|
||||||
|
|||||||
@ -1,7 +1,7 @@
|
|||||||
# Coverage Floor
|
# Coverage Floor
|
||||||
|
|
||||||
CI enforces three coverage gates on `workspace-server` (Go). All defined in
|
CI enforces coverage gates on two surfaces — `workspace-server` (Go) and
|
||||||
`.github/workflows/ci.yml` → `platform-build` job.
|
`workspace/` (Python). All defined in `.github/workflows/ci.yml`.
|
||||||
|
|
||||||
## Current floors (2026-04-23)
|
## Current floors (2026-04-23)
|
||||||
|
|
||||||
@ -76,3 +76,51 @@ This gate makes "no untested critical paths merged" a mechanical property of
|
|||||||
the CI, not a behavioural property of QA agents or individual reviewers —
|
the CI, not a behavioural property of QA agents or individual reviewers —
|
||||||
which is the only way to make it survive fleet outages, agent rotations, or
|
which is the only way to make it survive fleet outages, agent rotations, or
|
||||||
QA process changes.
|
QA process changes.
|
||||||
|
|
||||||
|
## Python (workspace/) — added 2026-05-04 from #2790
|
||||||
|
|
||||||
|
The Python side has its own gates in the `python-lint` job:
|
||||||
|
|
||||||
|
| Gate | Threshold | Where |
|
||||||
|
|---|---|---|
|
||||||
|
| **Total floor** | `86%` | `workspace/pytest.ini` `--cov-fail-under=86` (issue #1817) |
|
||||||
|
| **Critical-path per-file floor** | `75%` | Inline shell step after the pytest run |
|
||||||
|
|
||||||
|
### Critical-path Python files
|
||||||
|
|
||||||
|
These handle multi-tenant routing, auth tokens, and inbox dispatch. A
|
||||||
|
coverage drop here is the same risk shape as a Go-side `tokens*` /
|
||||||
|
`secrets*` file regressing below 10%.
|
||||||
|
|
||||||
|
- `workspace/a2a_mcp_server.py` — MCP dispatcher (PR #2766 / #2771)
|
||||||
|
- `workspace/mcp_cli.py` — molecule-mcp standalone CLI entry
|
||||||
|
- `workspace/a2a_tools.py` — workspace-scoped tool implementations
|
||||||
|
- `workspace/inbox.py` — multi-workspace inbox + per-workspace cursors
|
||||||
|
- `workspace/platform_auth.py` — per-workspace token resolver
|
||||||
|
|
||||||
|
### Why 75% (vs 86% total)
|
||||||
|
|
||||||
|
The total floor averages ~6000 lines across `workspace/`. A single MCP
|
||||||
|
file could drop to ~50% with no CI complaint as long as other modules
|
||||||
|
compensate. The per-file floor closes that distribution gap. 75% sits
|
||||||
|
below current actuals (80–96% as of 2026-05-04) — strictly additive,
|
||||||
|
no existing PR fails.
|
||||||
|
|
||||||
|
### Python ratchet plan
|
||||||
|
|
||||||
|
| Date | Total | Per-file critical | Notes |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 2026-05-04 | 86% | 75% | Initial gate (this file). |
|
||||||
|
| 2026-06-04 | 86% | 80% | First ratchet — at-floor files must catch up. |
|
||||||
|
| 2026-07-04 | 88% | 85% | |
|
||||||
|
| 2026-08-04 | 90% | 90% | Target steady-state. |
|
||||||
|
|
||||||
|
### Why this Python gate exists
|
||||||
|
|
||||||
|
Issue #2790, after the PR #2766 → PR #2771 cycle. PR #2766 added
|
||||||
|
multi-workspace routing through `a2a_tools.py` + `a2a_mcp_server.py`,
|
||||||
|
shipped to main with green CI, but the dispatcher silently dropped a
|
||||||
|
load-bearing kwarg for 4 of 9 tools — caught only by post-merge code
|
||||||
|
review. The structural drift gate (`test_dispatcher_schema_drift.py`,
|
||||||
|
PR #2791) catches the schema↔dispatcher mismatch class; this floor
|
||||||
|
catches the broader "MCP-critical file regressed" class.
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user