molecule-core/tools/check-template-parity.sh
Hongming Wang a56b765b2d
docs: testing strategy + PR hygiene + backend parity matrix + boot-event postmortem (#1824)
Bundles the documentation and lightweight tooling landed during the
2026-04-23 ops/triage session. Pure additions — no behavior changes.

## Added

### docs/architecture/backends.md
Parity matrix for Docker vs EC2 (SaaS) workspace backends. 18 features
tabulated with current status; 6 ranked drift risks; enforcement
hooks (parity-lint + contract tests). Living document — owners are
workspace-server + controlplane teams.

### docs/engineering/testing-strategy.md
Tiered test-coverage floors instead of a blanket 100% target. Seven
tiers by code class (auth/crypto → generated DTOs). Per-package
current-state snapshot + targets. Tracks the 3 biggest coverage gaps
(tokens.go 0%, workspace_provision.go 0%, wsauth ~48%) against their
tier-1/2 floors.

### docs/engineering/pr-hygiene.md
Captures the patterns that keep diffs reviewable. Motivated by the
2026-04-23 backlog audit where 8 of 23 open PRs had 70-380-file bloat
from stale branch drift. Covers: small-PR sizing, rebase-not-merge,
cherry-pick-onto-fresh-base for recovery, targeting staging first,
describing why-not-what.

### docs/engineering/postmortem-2026-04-23-boot-event-401.md
Postmortem for the /cp/tenants/boot-event 401 race. Root cause (DB
INSERT ordered AFTER readiness check), detection path (E2E + manual
log inspection), lessons (write-before-read pattern, integration
tests needed, E2E alerting gap, invariants-as-comments).

### tools/check-template-parity.sh
CI lint for template repos — diffs the `${VAR:+VAR=${VAR}}` provider-
key forwarders between install.sh (bare-host / EC2 path) and start.sh
(Docker path). Catches the #5 drift risk from backends.md before it
ships.

### workspace-server/internal/provisioner/backend_contract_test.go
Shared behavioral contract scaffold for Provisioner + CPProvisioner.
Compile-time assertions catch method-signature drift today; scenario-
level runs are t.Skip'd pending backend nil-hardening (drift risk #6,
see backends.md).

## Updated

### README.md
Links the new engineering docs + backends parity matrix into the
Documentation Map so agents and humans can actually find them.

## Related issues

- #1814 — unblock workspace_provision_test.go (broadcaster interface)
- #1813 — nil-client panic hardening (drift risk #6)
- #1815 — Canvas vitest coverage instrumentation
- #1816 — tokens.go 0% → 85%
- #1817 — 5 sqlmock column-drift failures
- #1818 — Python pytest-cov setup
- #1819 — wsauth middleware coverage gap
- #1821 — tiered coverage policy (meta)
- #1822 — backend parity drift tracker

Co-authored-by: Hongming Wang <hongmingwang.rabbit@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: molecule-ai[bot] <276602405+molecule-ai[bot]@users.noreply.github.com>
2026-04-23 19:59:38 +00:00

81 lines
2.7 KiB
Bash
Executable File

#!/usr/bin/env bash
# check-template-parity.sh — enforce parity between a workspace template's
# install.sh (bare-host / EC2 path) and start.sh (Docker path). Both scripts
# must forward the same set of provider API keys to the agent's .env so that
# a workspace built on one backend behaves identically to a workspace built
# on the other.
#
# Drift this catches:
# - Someone adds HERMES_API_KEY to start.sh but forgets install.sh.
# EC2 workspaces using Nous fail silently; Docker works.
# - Someone adds a HERMES_CUSTOM_BASE_URL branch to install.sh only.
# Docker can't use a custom OpenAI-compat endpoint; EC2 can.
#
# Invocation (from template-hermes repo's CI):
#
# bash /path/to/molecule-monorepo/tools/check-template-parity.sh \
# install.sh start.sh
#
# Or inline via curl:
#
# bash <(curl -fsSL https://raw.githubusercontent.com/Molecule-AI/molecule-core/main/tools/check-template-parity.sh) \
# install.sh start.sh
#
# Exit codes:
# 0 — parity ok (or both files declare the same set of ${VAR:+VAR=...} exports)
# 1 — drift detected (emits a diff to stderr)
# 2 — usage / missing files
#
# What "parity" means here: the SET of environment-variable forwarders
# (lines of the form `${VAR:+VAR=${VAR}}`) in each file must be equal.
# The ordering, surrounding comments, and non-forwarder lines are free to
# differ — that's where the two paths legitimately diverge (bare-host vs
# Docker-entrypoint structure).
set -euo pipefail
if [ "$#" -ne 2 ]; then
echo "usage: $0 install.sh start.sh" >&2
exit 2
fi
INSTALL_SH="$1"
START_SH="$2"
for f in "$INSTALL_SH" "$START_SH"; do
if [ ! -f "$f" ]; then
echo "missing file: $f" >&2
exit 2
fi
done
# Extract the set of ${VAR:+VAR=...} forwarder lines, stripped of
# surrounding whitespace. sort -u gives us the set to compare.
extract_forwarders() {
grep -oE '\$\{[A-Z_]+:\+[A-Z_]+=\$\{[A-Z_]+\}\}' "$1" 2>/dev/null | sort -u
}
TMP_INSTALL=$(mktemp)
TMP_START=$(mktemp)
trap 'rm -f "$TMP_INSTALL" "$TMP_START"' EXIT
extract_forwarders "$INSTALL_SH" > "$TMP_INSTALL"
extract_forwarders "$START_SH" > "$TMP_START"
if diff -q "$TMP_INSTALL" "$TMP_START" > /dev/null; then
COUNT=$(wc -l < "$TMP_INSTALL" | tr -d ' ')
echo "template-parity: ok ($COUNT provider forwarders in both files)"
exit 0
fi
echo "template-parity: DRIFT detected between $INSTALL_SH and $START_SH" >&2
echo >&2
echo "--- forwarders only in $INSTALL_SH ---" >&2
comm -23 "$TMP_INSTALL" "$TMP_START" | sed 's/^/ /' >&2
echo "--- forwarders only in $START_SH ---" >&2
comm -13 "$TMP_INSTALL" "$TMP_START" | sed 's/^/ /' >&2
echo >&2
echo "Fix: copy the missing forwarder lines so both files carry the same set." >&2
echo "Rationale: workspace-backend parity — see docs/architecture/backends.md" >&2
exit 1