molecule-core/.github/workflows/ci.yml
rabbitblood c4bb325267 ci(platform-go): add critical-path coverage gate + per-file report (#1823)
## Problem

External audit flagged critical security-path files at 0% coverage:
  - workspace-server/handlers/tokens.go            0%  (target 90%+)
  - workspace-server/handlers/workspace_provision  0%  (target 75%+)
  - workspace-server/middleware/wsauth            ~48% (target 90%+)

Tests *exist* for these files (tokens_test.go is 200 lines, workspace_
provision_test.go is 1138 lines) — they just don't exercise the critical
branches where auth/provisioning decisions happen. CI's existing coverage
step measured total coverage (floor 25%) but never checked per-file,
so any single file could drop to 0% and CI stayed green.

## Fix — Layer 1 of #1823 (strictly additive)

1. **Per-file coverage report** — advisory step prints every source file
   with its coverage, sorted worst-first. Reviewers see the gap at a
   glance. Does not fail the build.

2. **Critical-path per-file gate** — if any non-test source file in a
   security-sensitive directory (tokens, workspace_provision, a2a_proxy,
   registry, secrets, wsauth, crypto) has coverage ≤10%, CI fails with
   a specific error message pointing at the file + #1823.

3. **Unchanged: total floor stays at 25%** — ratcheting is a separate PR
   so this one has zero risk of breaking existing coverage. Ratchet plan
   lives in COVERAGE_FLOOR.md (monthly schedule through Oct 2026 to reach
   70% total / 70% critical).

## Why this specifically

"Tell devs to write tests" doesn't fix this — the prompts already
require tests ("Write tests for every handler, every query, every edge
case"), and the engineers mostly do. The gap is mechanical: CI generates
coverage.out and throws it away without checking per-file distribution.

This gate makes "no untested security path merges" a property of the CI,
not a property of QA agents who (as of today's incident) can go phantom-
busy for hours.

## Smoke test

Local awk-logic verification with synthetic coverage.out:
  - tokens.go at 2.5% (critical path, ≤10%)           → correctly FAILS
  - noncritical.go at 0.0% (not in critical list)     → correctly PASSES
  - wsauth_middleware.go at 65% (critical, above 10%) → correctly PASSES
  - crypto/kek.go at 85% (critical, above 10%)        → correctly PASSES

Regex bug caught and fixed: go tool cover -func emits
  file.go:LINE.COL:FUNC  PERCENT
The stripper needed :[0-9]+\..* not :[0-9]+:.*

## Follow-up (not in this PR)

- Layer 2 (issue #1823): per-changed-file delta gate via diff-cover,
  enforcing the prompt rule ">80% on changed files"
- Add these two new steps to branch protection required checks
- Canvas (Next.js) equivalent with vitest --coverage + threshold

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:12:40 -07:00

262 lines
11 KiB
YAML
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

name: CI
on:
push:
branches: [main, staging]
pull_request:
branches: [main, staging]
# Cancel in-progress CI runs when a new commit arrives on the same ref.
# This prevents stale runs from queuing behind each other.
concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true
jobs:
# Detect which paths changed so downstream jobs can skip when only
# docs/markdown files were modified.
changes:
name: Detect changes
runs-on: ubuntu-latest
outputs:
platform: ${{ steps.check.outputs.platform }}
canvas: ${{ steps.check.outputs.canvas }}
python: ${{ steps.check.outputs.python }}
scripts: ${{ steps.check.outputs.scripts }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- id: check
run: |
# For PR events: diff against the base branch (not HEAD~1 of the branch,
# which may be unrelated after force-pushes). When a push updates a PR,
# both pull_request and push events fire — prefer the PR base so that
# the diff is always computed against the actual merge base, not the
# previous SHA on the branch which may be on a different history line.
BASE="${GITHUB_BASE_REF:-${{ github.event.before }}}"
# GITHUB_BASE_REF is set by GitHub for PR events (the base branch name).
# For pull_request events we use the stored base.sha; for push events
# (or when base.sha is unavailable) fall back to github.event.before.
if [ "${{ github.event_name }}" = "pull_request" ] && [ -n "${{ github.event.pull_request.base.sha }}" ]; then
BASE="${{ github.event.pull_request.base.sha }}"
fi
# Fallback: if BASE is empty or all zeros (new branch), run everything
if [ -z "$BASE" ] || echo "$BASE" | grep -qE '^0+$'; then
echo "platform=true" >> "$GITHUB_OUTPUT"
echo "canvas=true" >> "$GITHUB_OUTPUT"
echo "python=true" >> "$GITHUB_OUTPUT"
echo "scripts=true" >> "$GITHUB_OUTPUT"
exit 0
fi
DIFF=$(git diff --name-only "$BASE" HEAD 2>/dev/null || echo ".github/workflows/ci.yml")
echo "platform=$(echo "$DIFF" | grep -qE '^workspace-server/|^\.github/workflows/ci\.yml$' && echo true || echo false)" >> "$GITHUB_OUTPUT"
echo "canvas=$(echo "$DIFF" | grep -qE '^canvas/|^\.github/workflows/ci\.yml$' && echo true || echo false)" >> "$GITHUB_OUTPUT"
echo "python=$(echo "$DIFF" | grep -qE '^workspace/|^\.github/workflows/ci\.yml$' && echo true || echo false)" >> "$GITHUB_OUTPUT"
echo "scripts=$(echo "$DIFF" | grep -qE '^tests/e2e/|^scripts/|^\.github/workflows/ci\.yml$' && echo true || echo false)" >> "$GITHUB_OUTPUT"
platform-build:
name: Platform (Go)
needs: changes
if: needs.changes.outputs.platform == 'true'
runs-on: ubuntu-latest
defaults:
run:
working-directory: workspace-server
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: 'stable'
- run: go mod download
- run: go build ./cmd/server
# CLI (molecli) moved to standalone repo: github.com/Molecule-AI/molecule-cli
- run: go vet ./...
- name: Run golangci-lint
uses: golangci/golangci-lint-action@v9
with:
version: latest
working-directory: workspace-server
args: --timeout 3m
continue-on-error: true # Warn but don't block until codebase is clean
- name: Run tests with race detection and coverage
run: go test -race -coverprofile=coverage.out ./...
- name: Per-file coverage report
# Advisory — lists every source file with its coverage so reviewers
# can see at-a-glance where gaps are. Sorted ascending so the worst
# offenders float to the top. Does NOT fail the build; the hard
# gate is the threshold check below. (#1823)
run: |
echo "=== Per-file coverage (worst first) ==="
go tool cover -func=coverage.out \
| grep -v '^total:' \
| awk '{file=$1; sub(/:[0-9]+\..*/, "", file); pct=$NF; gsub(/%/,"",pct); s[file]+=pct; c[file]++}
END {for (f in s) printf "%6.1f%% %s\n", s[f]/c[f], f}' \
| sort -n
- name: Check coverage thresholds
# Enforces two gates from #1823 Layer 1:
# 1. Total floor (unchanged at 25% this PR — ratchet plan in
# COVERAGE_FLOOR.md). Keeping it where it was keeps this PR
# strictly additive — the NEW protection is gate 2.
# 2. Per-file zero-floor — any .go file (non-test) in a
# security-critical path with coverage ≤10% fails the build.
# Catches the exact case that triggered #1823 (tokens.go at 0%).
run: |
set -e
TOTAL_FLOOR=25
# Files/paths that cannot drop to 0% coverage. Add here carefully;
# this is the "protected paths" list for security-sensitive code.
CRITICAL_PATHS=(
"internal/handlers/tokens"
"internal/handlers/workspace_provision"
"internal/handlers/a2a_proxy"
"internal/handlers/registry"
"internal/handlers/secrets"
"internal/middleware/wsauth"
"internal/crypto"
)
TOTAL=$(go tool cover -func=coverage.out | grep '^total:' | awk '{print $3}' | sed 's/%//')
echo "Total coverage: ${TOTAL}%"
if awk "BEGIN{exit !($TOTAL < $TOTAL_FLOOR)}"; then
echo "::error::Total coverage ${TOTAL}% is below the ${TOTAL_FLOOR}% floor. See COVERAGE_FLOOR.md for ratchet plan."
exit 1
fi
# Gate 3: critical files must not be 0%
FAILED=0
go tool cover -func=coverage.out \
| grep -v '^total:' \
| awk '{file=$1; sub(/:[0-9]+\..*/, "", file); pct=$NF; gsub(/%/,"",pct); s[file]+=pct; c[file]++}
END {for (f in s) printf "%s %.1f\n", f, s[f]/c[f]}' \
> /tmp/perfile.txt
for path in "${CRITICAL_PATHS[@]}"; do
while read -r file pct; do
if [[ "$file" == *"$path"* ]] && [[ "$file" != *_test.go ]]; then
if awk "BEGIN{exit !($pct < 10)}"; then
echo "::error file=workspace-server/$file::Critical file at ${pct}% coverage — must be >=10% (target 80%). See #1823."
FAILED=1
fi
fi
done < /tmp/perfile.txt
done
if [ "$FAILED" -eq 1 ]; then
echo ""
echo "One or more security-critical files have ≤10% test coverage."
echo "These paths handle auth, tokens, secrets, or workspace provisioning —"
echo "a 0% file here is the exact gap that let CWE-22, CWE-78, KI-005 slip"
echo "through in past incidents. Add tests or document an exception in"
echo "COVERAGE_FLOOR.md with a linked issue and 14-day expiry."
exit 1
fi
canvas-build:
name: Canvas (Next.js)
needs: changes
if: needs.changes.outputs.canvas == 'true'
runs-on: ubuntu-latest
defaults:
run:
working-directory: canvas
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '22'
- run: rm -f package-lock.json && npm install
- run: npm run build
- name: Run tests
run: npx vitest run
# MCP Server + SDK removed from CI — now in standalone repos:
# - github.com/Molecule-AI/molecule-mcp-server (npm CI)
# - github.com/Molecule-AI/molecule-sdk-python (PyPI CI)
# e2e-api job moved to .github/workflows/e2e-api.yml (issue #458).
# It now has workflow-level concurrency (cancel-in-progress: false) so
# new pushes queue the E2E run rather than cancelling it at the run level.
shellcheck:
name: Shellcheck (E2E scripts)
needs: changes
if: needs.changes.outputs.scripts == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run shellcheck on tests/e2e/*.sh
# shellcheck is pre-installed on ubuntu-latest runners (via apt).
run: |
find tests/e2e -type f -name '*.sh' -print0 \
| xargs -0 shellcheck --severity=warning
canvas-deploy-reminder:
name: Canvas Deploy Reminder
runs-on: ubuntu-latest
needs: [changes, canvas-build]
# Only fires on direct pushes to main (i.e. after staging→main promotion).
if: needs.changes.outputs.canvas == 'true' && github.event_name == 'push' && github.ref == 'refs/heads/main'
permissions:
# Required to post commit comments via the GitHub API.
contents: write
steps:
- name: Post deploy reminder as commit comment
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
COMMIT_SHA: ${{ github.sha }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
run: |
# Write body to a temp file — avoids backtick escaping in shell.
cat > /tmp/deploy-reminder.md << 'BODY'
## Canvas build passed ✅ — deploy required
The `publish-canvas-image` workflow is now building a fresh Docker image
(`ghcr.io/molecule-ai/canvas:latest`) in the background.
Once it completes (~35 min), apply on the host machine with:
```bash
cd <runner-workspace>
git pull origin main
docker compose pull canvas && docker compose up -d canvas
```
If you need to rebuild from local source instead (e.g. testing unreleased
changes or a new `NEXT_PUBLIC_*` URL), use:
```bash
docker compose build canvas && docker compose up -d canvas
```
BODY
printf '\n> Posted automatically by CI · commit `%s` · [build log](%s)\n' \
"$COMMIT_SHA" "$RUN_URL" >> /tmp/deploy-reminder.md
gh api \
--method POST \
"repos/${{ github.repository }}/commits/${{ github.sha }}/comments" \
--field "body=@/tmp/deploy-reminder.md"
python-lint:
name: Python Lint & Test
needs: changes
if: needs.changes.outputs.python == 'true'
runs-on: ubuntu-latest
env:
WORKSPACE_ID: test
defaults:
run:
working-directory: workspace
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: pip
cache-dependency-path: workspace/requirements.txt
- run: pip install -r requirements.txt pytest pytest-asyncio pytest-cov
- run: python -m pytest --tb=short -q --cov=. --cov-report=term-missing
# SDK + plugin validation moved to standalone repo:
# github.com/Molecule-AI/molecule-sdk-python